Robots Learn Faster with Quantum Technology

An international team of researchers have experimentally proved that quantum computing can provide a measurably significant speed-up to robots’ learning time.

“We have shown, for the first time, a learning task — or more precisely a reinforcement learning technique — that exploits the advantages of quantum technology for the machine or robot that learns a task, and for the communication with the environment that provides the feedback for the robot’s action,” explained Philip Walther, a professor of physics at University of Vienna who was the principal investigator in this research, leading a team of researchers from  University of Innsbruck, the Austrian Academy of Sciences, the Leiden University, and the German Aerospace Center. “Thus, we show a machine, where quantum technology is ‘maximally exploited.’ Our experiment shows that there is a speed-up compared to classical machine learning counterparts.”

The field of robotics has progressed tremendously in the last few years, in tandem with machine learning technologies. This has allowed for the development of robots that can “learn” like humans, whether it’s learning through tactile reasoning, or imitation learning through virtual reality or using deep learning to help a robot develop a way to “visualize” the future, or learn from its mistakes, much like a human would.

Of course, as these learning tasks become more complex, better hardware and data processing strategies will be needed to handle these more intensive computations. Here’s where the powerful potential of quantum computing could give machine learning an enormous boost. That’s because unlike a conventional, “classical” computer which processes information in the binary sense of 1s and 0s, a quantum computer leverages a quantum mechanical property called superposition, which allows it to process information as either or both 1s and 0s — all at the same time. This mind-boggling possibility of multistate information processing is what makes quantum computing so potentially revolutionary in fields like cryptography, medicine, finance, and of course, in accelerating machine learning.

Quantum Algorithms and Quantum Light

In findings that were published in Nature, the team describes how their experiments used fundamental particles of light called photons, in conjunction with an integrated, programmable nanophotonic quantum processor developed at MIT. This processor was used as a stand-in for a robotic agent, capable of implementing learning tasks.

“We used quantum light — a single photon — to be processed through a nanophotonic processor,” said Walther. “In a nutshell, the light enters from the ‘left side’ is then processed quantum mechanically, which means that this photon is superimposed and thus can travel through different paths simultaneously, before escaping the chip at ‘the right side.'”

In practical terms, the experiment was set up with the nanophotonic chip designated into different sections in order to more accurately monitor the experimental results, as the chip processed quantum algorithms and quantum light. “The first section is the robot, which can tune its processor part for guiding the photon. The next section is the environment that affects the photon with respect to its path, and as the last step — which we implemented as a new run — is the evaluation of the environment’s influence of the photon path.”

This last step was done using a classical computation that results in a “policy update”, or the process of acquiring new “knowledge” for the robot. According to the team, one can think of the experiment with the analogy of an intelligent robotic agent standing at an intersection and tasked with the learning of always turning left. When this left turn is correctly achieved, it receives a reward, which further reinforces the robot’s learning process. When performed under a classical computational system, the reward would only be obtained when the robot turns left. However, when done under a quantum mechanical paradigm, the quantum robot can prepare the plausible paths simultaneously, allowing it to learn much faster. In the case of the team’s experiments, the use of quantum computing technology reduced learning times for the robotic agent by 63%, down from 270 attempts to only 100.

Diagram of a robotic learning agent. A) Agent interacts with environment by receiving input (Si) and outputs consequent actions (ai). In the case of the correct action being chosen, the environment gives a reward (r), which enhances the agent’s performance in the next round. B) Agent-environment interacts via a “classical” channel, where communication in either one of two states is possible. C) Agent-environment interacts via quantum channel, where various superposition states can be communicated.

“We were surprised how nicely this quantum reinforcement learning can be optimized by allowing also classical runs, without the exploit of quantum features,” said Walther. “The reason for that is that the underlying algorithm that allows the robot to learn quickly is the so-called Grover’s search algorithm, a well-established algorithm that allows a quadratic speed-up for searching something.”

However, Walther notes that Grover’s search algorithm presents some intrinsic challenges, namely that using this algorithm not only increases the probability of finding a search target quickly, but also the probability of overshooting and therefore lower search success when the algorithm is run too long.

“In our experiment, we address this issue by starting the machine learning task quantum mechanically, and then at the end, we switch to a classical algorithm such that we can have a smooth convergence to the correct finding,” he added.

Ultimately, that the team’s demonstration shows that quantum technologies will undoubtedly help to accelerate the machine learning processes underpinning autonomous robots, and the benefits of implementing photonic architectures to do so, among other options.

“In the future, one can increase the complexity and size of such quantum machine learnings by using more photons and more complex interferometric networks,” said Walther. “However, our demonstration is not limited to photonic systems, as other types of quantum systems are also currently implementing quantum machine learning protocols.”

Read more in Nature.