Molecules, which are collections of tightly bound atoms, are all around in everyday life. For example, you can read this article because the light impinging on your retina drives a molecule called retinaldehyde to bend into a new 3D structure. This fine-tuning in the molecule's shape triggers a series of reactions, at the end of which an electric impulse reaches your brain and you can 'see' the object. The huge variety of molecular shapes and properties has inspired scientists to search for applications: molecules could build either tiny human-made 'molecular machines' which mimic processes in nature, or little molecular electronic devices and computers, especially quantum computers.

Such quantum computers – which are currently under development worldwide – could one day solve immensely complex problems (e.g. improving weather forecasts). However, the ominous 'quantum states' – which these computers use for calculations – are extremely fragile and could break very quickly when a disorder in the computer's building blocks occurs. So, if we want to build a quantum computer from individual molecules, we need to be able to arrange them with the highest precision. In a new study, we sought to develop new technology to achieve precise control of single molecules.

Fortunately, there is an instrument which at least in principle is up to the task of picking up and placing individual molecules: the low-temperature scanning probe microscope. At its core, the microscope has a sharp metal needle that can move across an object's surface, scanning its architecture with extreme precision. This vaguely resembles a classical record player for vinyl records. This microscope creates images by scanning the surface, but the needle also can bind chemically to a single molecule on the surface, move it around and release it in the right position.

The central problem of arranging molecules with this microscope is lack of vision: the needle can simultaneously execute only either task – imaging or moving the molecule on the object's surface. In other words, we can't watch what is actually happening to the molecule while moving it. Bringing a molecule in the right position by controlling the needle within our hands is therefore nearly impossible. To address this challenge, we developed a new approach, where artificial intelligence becomes the 'director' that does not require vision or any other human intervention for controlling the needle.

To test this concept, we carried out the task of detaching individual PTCDA molecules – known as organic dye molecules – from a closed layer. Since the molecules spontaneously order themselves in a checkerboard pattern when put on the surface, the hole created by the detachment of a molecule is aligned with the original 'grid' of the checkerboard pattern. Such an ordered molecular pattern could be useful in building a quantum computer. However, selectively detaching a single PTCDA molecule is hard since the molecules are tightly packed next to each other. Unless the needle is very precisely controlled, the bond between the needle and the prey molecule easily breaks during the manipulation.

Our artificial intelligence autonomously solves this problem of finding the right path in three dimensions using reinforcement learning, a type of machine learning. For reinforcement learning, every task is a game: an artificial intelligence performs actions and collects rewards – forming experiences – until the game is over. Based on this, it keeps trying to improve its strategy. After playing many, sometimes millions of rounds, the artificial intelligence may even outperform humans, say, in our beloved games like chess, Go or even curling.

As usual, our new artificial intelligence started with a blank experience memory by moving the microscope needle erratically back and forth and eventually 'failing' (breaking the bond to the molecule). This is the moment when the artificial intelligence receives a negative reward and starts all over again by approaching the needle back to the molecule to re-establish the bond. Our biggest challenge has been the low learning rate in the face of sparse feedback of 'success' or 'failure' – the minimal yes-or-no feedback that doesn't cover any possibly important details like how the prey molecule actually behaved during the task.

To overcome this problem, we instructed our artificial intelligence to develop a simple virtual model of its environment and gather additional experience by performing its own virtual removal experiments therein. This strategy is beneficial as such a virtual task takes place much faster than an actual task, which allows the artificial intelligence to train itself quickly, while the amount the artificial intelligence can learn through one task is not as high as an actual task. With the dual training strategy in reality and in the virtual model, our artificial intelligence quickly learned and succeeded in detaching a single molecule from the layer only after 10 - 20 attempts.

In summary, we showed that artificial intelligence could train itself quickly to perform tasks at the molecular-scale and with high accuracy. This innovation will help us to fabricate complex yet precise molecular buildings, which could be the foundation of future quantum computers.

Next read: Machine-learning boosts the conservation of endangered plant species by Tara Pelletier , Anahí Espíndola

More from Maths, Physics & Chemistry