Ep#94: Learning a Thousand Tasks in a Day
Show notes
For general-purpose robots to be useful, we must be able to teach them new tasks quickly and repurpose them to new roles. But most techniques for teaching robots — even from human demonstrations — take a great deal of new data. But it turns out that it is possible, in part through decomposing tasks into component subtasks, that it’s possible to learn a robot skills quickly.
Kamil Dreczkowski, Pietro Vitiello and Edward Johns join us to talk about their extensive study on how to teach robots new skills efficiently, while also generalizing to novel object instances, using a combination of task decomposition and retrieval, work that was published in Science Robotics.
Watch episode #94 of RoboPapers now, with Michael Cho and Jiafei Duan, to learn more!
Abstract
Humans are remarkably efficient at learning tasks from demonstrations, but today's imitation learning methods for robot manipulation often require hundreds or thousands of demonstrations per task. We investigate two fundamental priors for improving learning efficiency: decomposing manipulation trajectories into sequential alignment and interaction phases, and retrieval-based generalisation. Through 3,450 real-world rollouts, we systematically study this decomposition. We compare different design choices for the alignment and interaction phases, and examine generalisation and scaling trends relative to today's dominant paradigm of behavioural cloning with a single-phase monolithic policy. In the few-demonstrations-per-task regime (<10 demonstrations), decomposition achieves an order of magnitude improvement in data efficiency over single-phase learning, with retrieval consistently outperforming behavioural cloning for both alignment and interaction. Building on these insights, we develop Multi-Task Trajectory Transfer (MT3), an imitation learning method based on decomposition and retrieval. MT3 learns everyday manipulation tasks from as little as a single demonstration each, whilst also generalising to novel object instances. This efficiency enables us to teach a robot 1,000 distinct everyday tasks in under 24 hours of human demonstrator time. Through 2,200 additional real-world rollouts, we reveal MT3's capabilities and limitations across different task families. Videos of our experiments can be found on at this https URL.
Learn More
Project Page: https://www.robot-learning.uk/learning-1000-tasks
ArXiV: https://arxiv.org/abs/2511.10110
Science Paper: https://www.science.org/doi/abs/10.1126/scirobotics.adv7594
Github: https://github.com/kamil-dreczkowski/learning_thousand_tasks
This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com