Scalable Behavior Cloning with Open Data, Training, and Evaluation

Arthur Allshire, Himanshu Gaurav Singh, Ritvik Singh, Adam Rashid, Hongsuk Choi, David McAllister, Justin Yu, Yiyuan Chen, Huang Huang, Pieter Abbeel, Xi Chen, Rocky Duan, Phillip Isola, Jitendra Malik, Fred Shentu, Guanya Shi, Philipp Wu, Angjoo Kanazawa · · 2026

Summary

Abstract Summary

We introduce ABC, a fully open-source stack for manipulation with behavior cloning. At its core is ABC-130K: the largest open-source teleoperation dataset to date, featuring 3,500 hours of data spanning over 130K episodes across 195 diverse tasks. Furthermore, we open-source our accessible hardware setup, training infrastructure, and simulation pipeline. We also release 400 hours of sim-teleop data and provide a co-training recipe that produces correlated simulation and real-world evaluation, offering a reliable proxy for ablating model-design and training decisions before costly real-world evaluation. We explore various training recipes and compare common architectural choices for Diffusion Transformers (DiT) and Vision-Language-Action (VLA) models, grounding our findings in real-world evaluations. The resulting policies successfully execute dexterous tasks such as box folding and extracting credit cards from wallets. By providing a reproducible toolkit, we aim to place researchers on an equal footing, establishing the necessary foundation to learn the ABCs of Behavior Cloning together as a community.

Key Points

We introduce ABC, a fully open-source stack for manipulation with behavior cloning. At its core is ABC-130K: the largest open-source teleoperation dataset to date, featuring 3,500 hours of data spanning over 130K episodes across 195 diverse tasks.

ABC-130K Dataset

Publicly available robot data for policy learning has grown substantially in recent years. DROID [ 4 ] and BridgeData-V2 [ 6 ] provide high-quality single-arm data on the Franka and WidowX robots, respectively, but on tasks that are largely single-step pick-and-place.

Open X-Embodiment [ 5 ] spans many embodiments and tasks, but its wide variation in quality along with noisy language and action-space metadata makes it hard to use. AgiBot-World [ 7 ] offers 3,000 hours of bimanual data, although on a $30,000 robot.

ABC-130K Dataset — Scalable Behavior Cloning with Open Data, Training, and Evaluation — Figure 3 : Dataset overview. Random top camera frame samples from ABC-130K. We recommend zooming in to see the diverse tasks, backgrounds, and objects present in the data.

ABC-Models

ABC-DiT , a diffusion-transformer policy with lightweight vision and language encoders, and ABC-VLA , a VLM-backed policy with a diffusion action head. These two models reflect common design choices in recent robot learning systems: diffusion policies built on pretrained visual representations [ 9 , 10 ] , and vision-language-action models that adapt pretrained VLM backbones for robot control [ 11 , 12 , 13 , 14 ] .

ABC-Sim

$10$ $20$ $30$ $40$ $50$ $60$ $10$ $20$ $30$ $40$ $40$ $50$ $60$ $70$ $80$ $50$ $60$ $70$ $n=12$ $r=0.85$ , $p=4.2{\times}10^{-4}$ Sim strict success (%) Real strict success (%) Strict success $n=12$ $r=0.91$ , $p=5.0{\times}10^{-5}$ Sim task progress (%) Real task progress (%) Task progress Fit DiT VLA Figure 9 : Sim-to-real performance correlation.

Task-level simulation progress correlation with real-world progress across DiT and VLA checkpoints.

ABC-Sim — Scalable Behavior Cloning with Open Data, Training, and Evaluation — Figure 16 : Representative top-camera frames for each task category. Pick-and-Place (row 1, all 5): Packing luggage , Place fruits in bag , Organize toys on shelf , Place snacks in bag , Place glasses in tray . Fine Pick-and-Place (row 2, all 5): Build block tower , Load dishrack , Place coffee filter , Insert credit cards , Set up chess pieces . Folding (row 3, cells 1–4): Fold paper box , Fold skirt pile , Fold t-shirt pile , Roll up utensils . Insertion/Ejection (row 3 cell 5, row 4 cell 1): Attach microphone , Cap pens . Tool Use (row 4, cells 2–3): Lock with key , Seal zip-top bag . Sorting (row 4 cell 4): Organize lab equipment . Tying/Untying (row 4 cell 5): Tie flower bouquet .

Real-World Capabilities

Bottles Dishrack Mug flip $0$ $20$ $40$ $60$ $80$ $100$ Real progress (%) ABC-DiT ABC-VLA $0$ $50$ $100$ $150$ $200$ $0$ $25$ $50$ $75$ $100$ Checkpoint step (k) Sim progress (%) Bottles $0$ $50$ $100$ $150$ $200$ Checkpoint step (k) Dishrack $0$ $50$ $100$ $150$ $200$ Checkpoint step (k) Mug flip DiT 9.2kBS VLA 9.2kBS In this section, we demonstrate the capabilities of our trained policies on the publicly released ABC-130K set.

We begin by showing the performance of the pretrained base policies in Section 5.1 .

Real-World Capabilities — Scalable Behavior Cloning with Open Data, Training, and Evaluation — Figure 17 : Efficient frame access for fast data loading. Encoding keyframes more frequently and in a manner that allows for an analytically reconstructed frame index makes random frame access nearly free. To read one frame from a video, torchcodec ’s default scans the entire file to build its frame index (top). Correctly encoding the file allows us to compute the index analytically, meaning we only need to read the file header plus frames since the closest keyframe, leaving the rest of the file untouched, reducing disk pressure. The decoded frame is three vertically stacked camera views.

Infrastructure

Beyond the usual data and modeling details, the low-level systems implementations are equally important. To this end, we also release the surrounding infrastructure for this project. Appendix B.1 contains the details for efficient data loading from multiple video streams to improve training time. Appendix C details our lightweight and extensible real robot code framework.

Appendix D describes the inference optimizations for both ABC-DiT and ABC-VLA.

Infrastructure — Scalable Behavior Cloning with Open Data, Training, and Evaluation — Figure 20 : ABC-VLA pooled adaLN architecture. ABC-VLA uses a Gemma 3 VLM backbone, attention-pools the final VLM hidden states into a compact feature vector, and projects the result to an adaLN conditioning vector for the lightweight DiT action head.

Requests for Research

We present ABC-130K, ABC-Models, and ABC-Sim with the aim of seeding an open-source, reproducible ecosystem for robot learning research. Many interesting open questions remain to be answered for large-scale real-world robot learning, including but not limited to: History conditioning : policies that condition on a sequence of past observations rather than the current frame alone.

Scaling laws for BC : how performance scales with model and data size for imitation learning.

Requests for Research — Scalable Behavior Cloning with Open Data, Training, and Evaluation — (a)

Sources and Demonstrations

Figures are reproduced from Allshire et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.