UW Doctoral Student Researches How Robots Learn General Skills

To understand ourselves better, Roby Velez researches how robots learn general skills that help them explore their environment.

Specifically, the University of Wyoming doctoral student studies how a technique -- that encourages robots to explore the world in any way that is new -- causes them to learn basic skills of how to get around the world.

“The lab focuses on evolutionary algorithms, which are inspired by Darwinian evolution,” says Velez of Newark, N.J., who works with Jeff Clune, an assistant professor in the UW Department of Computer Science.

Such evolutionary algorithms are used in a field called “evolutionary computation,” which is a subset of the target field of artificial intelligence. An evolutionary algorithm uses mechanisms inspired by biological evolution, such as reproduction, mutation, recombination and selection.

While most evolutionary algorithms are designed to have a single, focused goal for a robot, Velez uses a technique called Novelty Search, in which the evolutionary algorithm is not trying to solve any specific task.

“With Novelty Search, there is no fixed goal or benchmark,” Velez explains in the Engineering Building’s Evolving Artificial Intelligence Lab. “The idea is, that if you evolve computer robots and let them be novel or different, you then can eventually get diverse sets of behaviors or outcomes. One of the diverse outcomes should be what you’re looking for.”

Gauging Fitness

During evolution in nature, humans and animals are rewarded when they find food to eat or mates with which they can reproduce. With evolutionary algorithms, Velez creates robots with artificial brains that are rewarded to get better at performing certain tasks or challenges.

For example, Velez says, “We want robot models that can walk long distances.”

A robot that can walk 12 feet -- opposed to another that can only walk 10 feet -- has a better chance of survival, Velez says.

“Based on how well they do, we give the robot a fitness score,” he says. “If the robot gets a good score, there is a relatively high chance it will pass its genes on to the next generation.”

Scores are relative to the population of robots, which can number 50 or 100, he says.

“When each is evaluated and scored, it’s scored against other robots,” Velez says. “Even if an individual robot gets a bad score, it doesn’t mean it will get killed off. The top robots have a 90 percent chance of passing on their genes. The bottom robots only have a 1 percent chance.”

Most advances in robotics today involve a lot of human effort, and engineers spend months writing software for each robot behavior, Clune says.

“But, there are too many things that we want robots to do to hand-code them all, so that manual approach will not scale,” Clune says. “Instead, we want to create robots that learn on their own how to accomplish tasks in the world. We want robots that explore their world, learn how to get around it and learn how to solve problems, just as animals and children do. Roby is studying how that might happen.”

Velez received his bachelor’s degree in engineering from Swarthmore College in Swarthmore, Pa., and his master’s degree in informatics from the University of Sussex in the United Kingdom. He says his informatics education embodied an approach to artificial intelligence that looked to nature for inspiration.

After he finished his master’s degree, Velez says he took six months off, but had signed up for a few email lists that dealt with artificial intelligence. In the span of a week, he received three emails about Clune’s work, and decided to look into the possibility of joining his lab.

Velez began pursuit of his doctoral degree at UW last August.

“Roby loves robotics and science. He has a knack for seeing the world from the robot’s perspective,” Clune says. “That enables him to think of different ways of solving difficult robotics problems because he comes at it from the robot’s point of view.”

Clune adds that Velez works hard and is passionate about sharing his work. For example, Velez has taken a lead role in the Laramie Robotics Club to teach local-area students a love for science, programming, robotics and math.

Eventually Scaling Up

Velez has not yet used Mount Moran, UW’s high-performance computing center, nicknamed after a mountain peak in western Wyoming’s Tetons. But, he says the need will arise when his computer robots become more complex and can do more complex tasks.

Mount Moran enables atmospheric and earth sciences faculty -- who will be able to use the NCAR-Wyoming Supercomputing Center (NWSC) -- to learn what to expect with their software. The cluster provides opportunities for that group of faculty to work out issues caused by scaling up parallel algorithms from tens or hundreds of processors to thousands of processors, before moving up to tens of thousands of processors on the NWSC.

The cluster also provides a research resource for any UW research faculty member -- such as bioinformaticists, social scientists, pure mathematicians and theoretical physicists – who has a complex problem or whose research doesn’t fall within the scope of the NWSC.

Additionally, UW students are welcome to use the high-performance computing center for their work, which is often in concert with UW faculty.

As a child, Velez says he liked working with his hands and playing with toys. He remembers taking parts from around the house and building cars. He says that early interest initially led to his pursuit of engineering for his undergraduate degree. As he continued his education, he decided to combine his interests in engineering, electronics and computers.

“I feel very strongly about the (artificial intelligence) field for a number of reasons. There’s still so much about emotions, morals and the conscience that we don’t understand in the human brain,” Velez says. “And, so, by trying to create an organism through artificial intelligence and robotics -- all the while collaborating with biologists, psychologists, philosophers and neuroscientists -- one of the hopes is to understand how we’re built, just understanding ourselves.”

“Curiosity is present in humans and, arguably, some animals,” Clune says. “There is a drive to do something new, whether intellectually, artistically, or simply to explore a new part of the world. Does that drive towards novelty mean that we become better explorers? This research addresses that question, albeit in the very different context of curious robots.”