How Trainable Learning Robots Can Be Even More Productive

Robots can work efficiently and effectively today, but are limited to operating within the parameters that have been programmed for them.

A new generation of robots capable of learning may be far more productive precisely because they can be trained to observe, react and anticipate. JL


Cade Metz reports in the New York Times:

“Today, every motion that an industrial robot makes is specified down to the millimeter. But most real problems can’t be solved that way. You have to be able not just to tell the robot what to do, but to tell it how to learn.” The hope is that robots can master a much wider array of tasks by learning on their own, learn(ing) to  install car parts that aren’t quite like the parts they installed in the past, sort through a bucket of random holiday gifts at a warehouse, or perform other tasks that machines traditionally could not.

During a recent speech at the University of California, Berkeley, Pieter Abbeel played a video clip of a robot doing housework.

In the clip recorded in 2008, the robot swept the floor, dusted the cabinets, and unloaded the dishwasher. At the end of it all, it even opened a beer and handed it to a guy on a couch.

The trick was that an engineer was operating the robot from afar, dictating its every move. But as Mr. Abbeel explained, the video showed that robotic hardware was nimble enough to mimic complex human behavior. It just needed software that could guide the hardware — without the help of that engineer.

“This is largely a computer science problem — an artificial intelligence problem,” Mr. Abbeel said. “We have the hardware that can do the job.”

Mr. Abbeel, a native of Belgium, has spent the last several years working on artificial intelligence, first as a Berkeley professor and then as a researcher at OpenAI, the lab founded by Tesla chief executive Elon Musk and other big Silicon Valley names. Now, he and three fellow researchers from Berkeley and OpenAI are starting their own company, intent on bringing a new level of robotic automation to the world’s factories, warehouses and perhaps even homes.

Photo

A robot at Embodied Intelligence that is learning how to manipulate wires. Credit Amy Harrity for The New York Times

Their start-up, Embodied Intelligence, is backed by $7 million in funding from the Silicon Valley venture capital firm Amplify Partners and other investors. The company will specialize in complex algorithms that allow machines to learn tasks on their own. Using these methods, existing robots could learn to, for example, install car parts that aren’t quite like the parts they have installed in the past, sort through a bucket of random holiday gifts as they arrive at a warehouse, or perform other tasks that machines traditionally could not.

“We now have teachable robots,” Mr. Abbeel said during a recent interview at the new company’s offices in Emeryville, Calif., just across the bay from San Francisco.

The new company is part of a much wider effort to create A.I. that allows robots to learn. Researchers in places like Google, Brown University, and Carnegie Mellon are doing similar work, as are existing start-ups like Micropsi and Prowler.io.

Robots already automate some work inside factories and warehouses, such as moving boxes from place to place at Amazon’s massive distribution centers. But companies must program these machines for each particular task, limiting their possible applications. The hope is that robots can master a much wider array of tasks by learning on their own.

“Today, every motion that an industrial robot makes is specified down to the millimeter,” said Sunil Dhaliwal, the Amplify founder who led the firm’s investment in Embodied Intelligence. “But most real problems can’t be solved that way. You have to be able not just to tell the robot what to do, but to tell it how to learn.”

Photo

Tianhao Zhang using virtual reality tools to show a robot how to perform a task. Credit Amy Harrity for The New York Times

Mr. Abbeel and the other founders of Embodied Intelligence, including the former OpenAI researchers Peter Chen and Rocky Duan and the former Microsoft researcher Tianhao Zhang, specialize in an algorithmic method called reinforcement learning — a way for machines to learn tasks by extreme trial and error.

Researchers at DeepMind, the London-based A.I. lab owned by Google, used this method to build a machine that could play the ancient game of Go better than any human. In essence, the machine learned to master this enormously complex game by playing against itself — over and over and over again.

Other researchers, across both industry and academia, have shown that similar algorithms allow robots to learn physical tasks as well. By repeatedly trying to open a door, for instance, a robot can learn which particular movements bring success and which don’t.

Much like Google and labs at Brown and Northeastern University, Embodied Intelligence is also augmenting these methods with a wide range of other machine learning techniques. Most notably, the start-up is exploring what is called imitation learning, a way for machines to learn discrete tasks from human demonstrations.

The company is using this method to teach a two-armed robot to pick up plastic pipes from a table. Donning virtual reality headsets and holding motion trackers in their hands, Mr. Abbeel and his colleagues will repeatedly demonstrate the task in a digital world that recreates what is in front of a robot. Then the machine can then learn from this digital data.

“We collect data on what the human is doing,” Mr. Chen said. “Then we can train the machine to imitate the human.”

These and similar machine learning methods have only begun to bear fruit over the past few years, but many believe they will overhaul the field of robotics. It is telling that OpenAI, a pure research lab that opened its doors less than two years ago, has now lost two big names to more commercial pursuits.

Poached away from OpenAI by Mr. Musk himself, the machine learning expert Andrej Karpathy is now the director of A.I. at Tesla, working to help autonomous cars get smarter faster. And Mr. Abbeel is launching Embodied Intelligence. “Andrej is stepping out of the research mode and saying: ‘I want to do this in practice,’” Mr. Abbeel explained. “That same thing goes for us.”

He said he believed his new start-up can rapidly push its methods into manufacturing operations like the auto industry. Although robotics already handle many tasks inside such factories, there are many others they can’t yet master. That is what Embodied Intelligence hopes to change.

Some researchers question how much these machine learning techniques will ultimately improve robotics, believing they are overhyped among both researchers and the news media. “Machine learning is being thrown at so many problems in robotics,” said Robert Howe, a professor of robotics at Harvard University. “And it produces just enough results that people can trumpet it.”

But Mr. Abbeel is among the world’s top researchers in his field, and his decision to start a own company is an indication that machine learning will continue to push robotics forward. “It is obvious that this is what you need to build flexible, agile robotics,” said Geoff Hinton, a pioneer of machine learning.