Robots Can Recover From Damage in Minutes, UW Researcher Helps Demonstrate

Robots will one day provide tremendous benefits to society, such as in search-and- rescue missions and putting out forest fires -- but not until they can learn to keep working if they become damaged.

Jeff Clune, a University of Wyoming assistant professor in the Department of Computer Science contributed to a paper, titled “Robots That Can Adapt Like Animals,” that shows how to make robots automatically recover from injury in less than two minutes. The paper appeared in today’s (May 28) issue of Nature, an international weekly journal of science that publishes the finest peer-reviewed research in all fields of science and technology.

Antoine Cully, lead author of the paper and a doctoral student at Pierre and Marie Curie University in France; and Jean-Baptiste Mouret, a then-assistant professor of artificial intelligence at Pierre and Marie Curie University, led the work. They collaborated with Clune and Danesh Tarapore, a then-doctoral student from Pierre and Marie Curie University. Tarapore is now a Marie Curie Research Fellow at the University of York in the United Kingdom.

In contrast to today’s robots, animals exhibit an amazing ability to adapt to injury. For example, there are many three-legged dogs that can catch Frisbees. If your ankle is sprained, you quickly figure out a way to walk despite the injury. The scientists took inspiration from these biological strategies.

“When injured, animals do not start learning from scratch,” says Mouret, the paper’s senior author. “Instead, they have intuitions about different ways to behave. These intuitions allow them to intelligently select a few, different behaviors to try out. And, after these tests, they choose one that works in spite of the injury. We made robots that can do the same.”

Before it is deployed, the robot uses a computer simulation of itself to create a detailed map of the space of high-performing behaviors. This map represents the robot’s “intuitions” about different behaviors it can perform and their predicted value. If the robot is damaged, it uses these intuitions to guide a learning algorithm that conducts experiments to rapidly discover a compensatory behavior that works despite the damage. The new algorithm is called “Intelligent Trial and Error.”

“Once damaged, the robot becomes like a scientist,” says Cully, the paper’s lead author. “It has prior expectations about different behaviors that might work, and begins testing them. However, these predictions come from the simulated, undamaged robot. It has to find out which of them work, not only in reality, but given the damage.

“Each behavior it tries is like an experiment and, if one behavior doesn’t work, the robot is smart enough to rule out that entire type of behavior and try a new type,” Cully continues. “For example, if walking, mostly on its hind legs, does not work well, it will next try walking mostly on its front legs. What’s surprising is how quickly it can learn a new way to walk. It’s amazing to watch a robot go from crippled and flailing around to efficiently limping away in about two minutes.”

The same Intelligent Trial and Error algorithm allows robots to adapt to unforeseen situations, including adapting to new environments and inventing new behaviors.

Clune explains that “technically, Intelligent Trial and Error involves two steps: (1) creating the behavior-performance map, and (2) adapting to an unforeseen situation.”

The map in the first step is created with a new type of evolutionary algorithm called MAP-Elites. Evolutionary algorithms simulate Darwinian evolution by hosting “survival of the fittest” competitions in computer simulations to evolve artificially intelligent robots. The adaptation in the second step involves a “Bayesian optimization” algorithm that takes advantage of the prior knowledge provided by the map to efficiently search for a behavior that works despite the damage.

“We performed experiments that show that the most important component of Intelligent Trial and Error is creating and harnessing the prior knowledge contained in the map,” Clune says.

This new technique will help develop more robust, effective, autonomous robots.

Tarapore provides some examples.

“It could enable the creation of robots that can help rescuers without requiring their continuous attention,” Tarapore says. “It also makes easier the creation of personal robotic assistants that can continue to be helpful even when a part is broken.”

This work was funded by the Agence Nationale pour la Recherche, the European Research Commission and a Direction Générale de l’Armement scholarship awarded to Cully.

---