Ep#110: ENPIRE: Agentic Robot PolicySelf-Improvement in the Real World

In many fields, automated research has become increasingly plausible with the improving capabilities of frontier language models, and the field robotics may be no exception. In ENPIRE, Wenli Xiao, Jia Xie, and Tonghe Zhang build a harness framework that captures physical feedback and allows for automated robotics research, minimizing human effort while training policies that achieve a 99% success rate on challenging, dexterous tasks like organizing a pin box or fastening a zip tie.

Learn more in Episode 110 of RoboPapers, with Michael Cho, Chris Paxton, and Ruijie He!

Abstract

Achieving dexterous robotic manipulation in the real world heavily relies on human supervision and algorithm engineering, which becomes a central bottleneck in the pursuit of general physical intelligence. Although emerging coding agents can generate code to automate algorithm search, their successes remain largely confined in digital environments. We conjecture that the missing abstraction to automate robotics research is a repeatable feedback loop for real-world policy improvement: reset the scene, execute a policy, verify the outcome, and refine the next iteration. To bridge this gap, we introduce ENPIRE, a harness framework for coding agents that instantiates this physical feedback routine with four core modules: an Environment module (EN) for automatic reset and verification, a Policy Improvement module (PI) that launches policy refinement, a Rollout module (R) to evaluate policies with one or multiple physical robots operating in parallel, and an Evolution module (E) in which coding agents analyze logs, consult literature, improve training infrastructure and algorithm code to address failure modes. This closed-loop system transforms real-world manipulation learning into a controllable optimization procedure, minimizing human effort while allowing fair ablations across training recipe and agent variants. Powered by ENPIRE, frontier coding agents can autonomously train a policy to achieve a 99% success rate on challenging, dexterous manipulation tasks, such as organizing a pin box, fastening a zip tie, and tool use, a process that further accelerates when we dispatch an agent team on a robot fleet. Our results suggest a practical and scalable path toward deploying coding agents to autonomously advancing robotics in the physical world.

Learn More

Project Page:

ArXiV: https://arxiv.org/abs/2606.19980

Transcript

Michael Cho (00:06.088)Hello everyone, welcome to another episode of RoboPapers. Today we’re super happy to have Wenli, Jia Xie and Tonghe. They are three of the co-authors of this paper called ENPIRE: Agentic Robot Policy Self-Improvement in the Real World. Thank you so much for coming to the pod. Maybe as a start three of you can share a little bit about yourself first and then you guys can go through the paper.

Wenli Xiao (00:30.821)Okay, I’ll get started. Sorry. So my name is Wenli. I’m a final-year PhD at CMU, advised by Professor Guanya Shi. Previously I did two-year internship at NVIDIA GEAR Lab with Jim Fan and Yuke Zhu. And later I joined Physical Intelligence as intern.

Yeah, I did a bunch of VLA training, post-training, real-world RL and agentic stuff and now I’m mainly doing pretraining, studying the science of training and data. Yeah.

Michael Cho (01:14.488)Yeah, I think just yeah.

Jia Xie (01:16.463)Okay, sure. Yeah. Hi everyone. My name is Jia and I’m a master’s student at CMU, advised by Guanya. Of course we’re labmates. And I’ve interned at xDof as a research intern over the summer. And I’ve been working on some agent stuff and the extent stuff lately. So yeah, thanks for having us.

Tonghe Zhang (01:43.739)Okay. Hi, my name is Tonghe. I’m also a master’s student at Professor Guanya Shi’s lab. I was doing internship at Amazon this summer and previously I’ve been working on reinforcement learning. And thank you guys for inviting us.

Michael Cho (02:01.15)Alright, perfect. Great to have you guys. Yeah, Wenli, I know you have a slide, maybe you can go through with us.

Wenli Xiao (02:08.228)Okay, yeah. So yeah, today the main topic is to introduce our paper, but as we all know, recently there’s a major breakthrough in AGI and that affects robotics, that is GPT-6 Astra. There are a lot of demos from Twitter that shows people apply this model to do different tasks.

That related to robotics, like real-to-sim, sim-to-real, in-context learning, and code-as-policy. And we are very excited and we are kind of a little bit frustrating and also wondering how well robotics been solved. But we will leave the discussion to later section. And because even before the GPT-6 Astra

Already witnessed a huge improvement of coding agent this year because we felt like coding agent is a biggest thing in AI this year because starting from February or last year’s December we see like a lot of a lot very good coding agent like Opus and Codex. They can do a lot of stuff like starting from front-end to

Back-end and build a lot of great applications. And actually, people have been trying to use these models in robotics every time there’s a breakthrough in AI. And for this year there’s a CaP-X project, which we apply the and benchmark the state-of-the-art LLM model and VLM model to

Try to solve and hill-climb on the robotic simulation benchmarks and it did pretty well in simulation. And we also tried to use coding agent to just solve the very challenging manipulation benchmark. This is a very classical task which would you need to manipulate a dot to push a T-shaped object until you perfectly matches the target.

Wenli Xiao (04:31.492)And this was really challenging and people find a solution or people didn’t find a solution until diffusion policy. But now we just prompt the coding agent out of the box. This is actually Codex. We prompted to write a heuristic policy with no neural network training to achieve a hundred percent success rate. It just try and try and eventually it can like

Write very good heuristic b policy. And this applies to different coding agents like Codex Claude Code and Kimi from Moonshot. This was very exciting to me. And actually during the past two years there are many impressive works about applying coding agent for robotics.

So here I just like briefly divide them into three categories, like VLA plus harness, so you can achieve great performance that can be better than either VLA or code-as-policy itself. And there’s of course Code-as-Policy, which people apply coding agent to try and practice in simulation so they can learn some skills and later apply to the real-world.

Real world robotics. And there’s also physical autoresearch, which is the ENPIRE work that we are gonna discuss today. So the major difference between the first two categories and the physical autoresearch is even all of them applied the method in the real-world. For the first two part of first two categories, they need tons of human effort.

So if we summarize the procedure of the researchers do these projects, it’s like they apply coding agent to write code and do some practice in simulation and also with some guidance of human, it develops some impressive skills. And then the researchers, PhD students, deploy the policy in the real-world and watch the results.

Jia Xie (06:54.974)Okay.

Wenli Xiao (06:55.682)And maybe for the first trial it doesn’t work, or even the robot just do nonsense. And then the student just describes what happened and type in the terminal, and then the coding agent fix it and do another trial error and apply and suddenly it works, and then can submit paper. So in this process we observe that humans are a major bottleneck.

Because we become the agent’s actuators and sensors. Yeah, because the observation was from like six months ago, now with GPT 6 extra, maybe we don’t need to be the sensors of the robot. GPT can like if you set up a webcam in a third-person view, maybe GPT can observe that the policy doesn’t work.

But six months ago we didn’t have such good GPT and we have GPT 5.5 and we still…

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com