Ep#78: Three Eras of Robot Learning

Robotics has changed dramatically over the last eight years. Ted has been involved in the cutting edge of robot learning through this period, spending those eight years at Google Brain/Google Deepmind. And he’s identified three eras of robot learning.

These eras are:

* The Era of Existence Proofs - trying different methods like QT-Opt, on-robot RL

* The Era of Foundation Models - transitioning to data collection and clean objectives (i.e. supervised learning)

* The Era of Scaling - orders of magnitude more data and larger models, enabling reasoning, long-horizon actions, and cross-embodiment transfer

The only reason something succeeds is if everything goes right. Behavior cloning, for example, seemed stuck at 60-70% success rate on key tasks until his team rewrote their learning stack — at which point it hit 95-99%+ success rates.

For most of those eight years, something was wrong. The stack wasn’t quite right, the learning algorithms were wrong, the data didn’t exist. Hardware and operations are not mature enough. But they kept working on these problems, over and over, until finally they have arrived at amazing breakthrough.

Some key trends now:

* Reasoning models for robotics

* Long-horizon, precision-oriented tasks, like making coffee from Physical Intelligence or GPU assembly from Skild

* Cross-embodiment transfer

* Hardware and model co-design

* Results are nice, but capabilities are even more — and academics are going to have trouble keeping up with compute and resources available to companies

Watch Episode 78 of RoboPapers, with Michael Cho and Jiafei Duan, to learn more!

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com