Co-Evolving Robot Orchestrators and Policies through Deployment

Xilun Zhang, Maggie Wang, Erik Bauer, Hong-Xing Yu, Huang Huang, Jiajun Wu, Marco Pavone · N/A · 2026

Summary

Abstract Summary

Vision-language-action (VLA) policies trained on large datasets are capable within their training domains, yet they still fail to generalize to the variety of situations a robot meets in real-world deployment. Agentic robot systems complement the policy with a vision-language model (VLM) orchestrator that learns when to call the policy, how to instruct it, and when to use scripted skills instead. However, because the harness is built around a frozen policy that has limited language steerability, the orchestrator can avoid the policy's failures but never overcome them. The policy becomes the bottleneck of the whole system. Fine-tuning the policy can remove this bottleneck, but updating it alone decouples it from an orchestrator tuned to its old behavior. We propose Robo-COP, in which the orchestrator and policy co-evolve during deployment. Robo-COP curates skill demonstrations from its own executions, fine-tunes the policy when this data can address recurring failures, and adopts each new policy only after it improves the skills it was trained for. Across ten simulated RoboLab tasks, Robo-COP raises mean held-out success from 64.8% to 73.8% over the same harness with a frozen policy, while fine-tuning on a fixed schedule without verification reaches only 65.8%. On three real-world tasks, Robo-COP raises held-out success from 38.3% to 50.0%. Robo-COP turns deployment into a self-improving flywheel in which robots learn by doing, with each improvement in execution producing better data for the next round of learning. Videos and code are available at https://robo-cop.pages.dev/.

Key Points

Vision-language-action (VLA) policies trained on large datasets are capable within their training domains, yet they still fail to generalize to the variety of situations a robot meets in real-world deployment.
Agentic robot systems complement the policy with a vision-language model (VLM) orchestrator that learns when to call the policy, how to instruct it, and when to use scripted skills instead.
However, because the harness is built around a frozen policy that has limited language steerability, the orchestrator can avoid the policy's failures but never overcome them.
The policy becomes the bottleneck of the whole system
Fine-tuning the policy can remove this bottleneck, but updating it alone decouples it from an orchestrator tuned to its old behavior.

Method

Robo-COP updates both the VLA policy and the adaptive harness that orchestrates its use (Figure 2 ). The harness logs policy and scripted actions as one stream (Section 3.1 ), curates it into skill demonstrations (Section 3.2 ), and verifies each new policy before adoption, revising the memory tied to the old one (Section 3.3 ).

Robo-COP is an embodied coding agent built on the CaP-X execution tools ( Fu et al., 2026 ) , extended with an execute_vla tool that invokes the VLA policy on a language instruction.

Method — Co-Evolving Robot Orchestrators and Policies through Deployment — Figure 3 : Skill-level data curation. Three recorded examples from Recycle cup clutter: both judges accept the pickup (A), the text judge accepts a grasp that the video judge rejects (B), and both reject a placement (C). The retained pickup comes from an episode that ultimately failed.

Experiments

Through this section, we aim to address three main questions. Q1: Does co-evolution improve held-out success over a frozen-policy harness and fixed-schedule training, and in which task regimes? Q2: Does the system decide correctly when to train and whether to adopt? Q3: Does learning during deployment cost task success along the way?

We answer them on ten simulated tasks and then test the full loop on three real-world tasks.

Experiments — Co-Evolving Robot Orchestrators and Policies through Deployment — Figure 4 : Representative RoboLab tasks. Object configurations vary across deployment and held-out evaluation.

Sources and Demonstrations

Figures are reproduced from Zhang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.