Robotics Research Hub
Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

Yen-Jen Wang, Haozhe Jiang, Shuying Deng, Haoru Xue, Weirui Ye, Rocky Duan, Nika Haghtalab, S. Shankar Sastry, Pieter Abbeel, Haozhi Qi · N/A · 2026

manipulation vision vla control benchmark
PDF arXiv Code (GitHub)

Framework

N/A

License

N/A

Stars

N/A

Summary

Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control.

Abstract Summary

Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control. We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights. RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation. During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures. It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses. Cross-task evaluation tests individual candidate changes and merged revisions before they are retained for reuse. At test time, a multimodal LLM uses the resulting system prompt and skill library to coordinate perception and robot control. On held-out initializations of 22 manipulation tasks, RPG improves task success from 28.6% after the first practice round to 95.0% after 15 rounds, outperforming all evaluated baselines, including ASPIRE (75.5%) and CaP-Agent0 powered by GPT-6 Astra Pro (60.0%). After a common calibration and hardware-adaptation procedure, the frozen system succeeds in all 30 physical trials, with ten trials on each of three tasks. Project Website: https://rpg-robot.github.io/

Key Points

  • Building reliable robot capabilities across diverse tasks requires substantial human effort to develop and maintain skills, design rewards, and integrate perception with control.
  • We present Reconstruct, Practice, Go Real (RPG), a framework for autonomous improvement of robot execution systems without updating model weights.
  • RPG identifies manipulation capabilities in an offline dataset and constructs related practice tasks in simulation.
  • During practice, RPG uses execution feedback, privileged simulator state, and available dataset videos to diagnose failures.
  • It develops new reusable symbolic skills, refines existing skills, and revises the system prompt based on these diagnoses.

Reconstruct, Practice, Go Real

RPG coordinates six specialized agents across its three stages: Constructor, Runtime Agent, Privileged Agent, Video Analyzer, Implementor, and Merger (Fig. 2 ). It first constructs simulation practice tasks from an offline dataset, improves a shared robot execution system by practicing these tasks, and ultimately deploys the resulting system on a physical robot.

Reconstruct, Practice, Go Real — Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents
Fig. 4: Task completion and revision activity. (A) Round-end systems are evaluated retrospectively on the same 22 tasks $\times$ 10 held-out seeds after development is frozen. Dashed lines show ASPIRE, RATs, and the matched-model and strongest CaP-Agent0 configurations (abbreviated CaP). Table II gives the full model comparison. (B) Stacked bars count accepted skill additions (green) and modifications (light blue) in each round. Purple squares mark rounds in which the Runtime Agent’s system prompt was revised (rounds 1–4); their height does not encode a count. The curve represents one improvement run. Rounds do not represent matched computational budgets.

Evaluation

Q1 , how does task success change with practice, and how does RPG compare with existing agents? Q2 , how do the diagnostic inputs affect improvement? Q3 , can library changes improve task completion with the surrounding runtime fixed? Q4 , does the resulting system complete physical tasks after common adaptation? The simulated YAM robot has two 6-DoF arms and two grippers, and receives RGB-D observations.

It accepts 14-dimensional commands specifying absolute joint-position targets.

Evaluation — Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents
Fig. 5: Physical task execution. Representative execution sequences on the physical YAM robot. The first three rows show the quantitatively evaluated tasks: Store Ball in Drawer, Fold Towel, and Transfer Bowl. Bowl transfer uses table-supported staging. The remaining rows show additional qualitative deployments on Insert Cylinder, Serve Fruit onto Plate, and Sort Utensils. Table V reports outcomes for the three quantitative tasks.

Sources and Demonstrations

Figures are reproduced from Wang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
arXiv preprint

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Haoran Chen, Jingtian Ji, Samuel Wheeler et al.

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.

manipulation vision vla reinforcement-learning control benchmark
PDF Intermediate
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
arXiv preprint

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

Yunchao Yao, Zhuxiu Xu, Tianqi Zhang et al.

Building general-purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
PDF Advanced