Open X-Embodiment: Robotic Learning Datasets and RT-X Models
FeaturedOpen X-Embodiment Collaboration · Google DeepMind and 20+ institutions worldwide · 2023
Framework
Jupyter Notebook
License
Apache-2.0
Stars
1,853
Summary
The largest collaborative robot learning dataset ever assembled, spanning 22 robots from 21 institutions, enabling training of generalist RT-X policies that exhibit positive cross-embodiment transfer.
Abstract Summary
Key Points
- Largest open robot learning dataset: 160k+ tasks and 527 skills across 22 robots and 21 institutions.
- Standardized data format enabling training of single policies on heterogeneous robot hardware.
- RT-X generalist policy shows positive transfer and outperforms specialist policies on held-out tasks.
- Action tokenization and cross-embodiment batching are key to multi-robot pre-training stability.
- Open-sourced dataset, baselines, and evaluation protocols to accelerate community research.
Additional Notes
Getting Started
- Clone the dataset repository and use the provided TFDS pipeline for efficient loading.
- For training RT-X from scratch, the authors recommend starting with pre-computed action tokenization.
- Evaluation scripts assume standard camera intrinsics; recalibrate if using custom hardware.
Related Papers
- RT-1 (Brohan et al., 2022)
- RT-2 (Brohan et al., 2023)
- Octo (Ghosh et al., 2024)
Related Papers
MonoDuo: Using One Robot Arm to Learn Bimanual Policies
Sandeep Bajamahal, Lawrence Yunliang Chen, Toru Lin et al. · ICRA 2026 · May 2026
Bimanual coordination is essential for many real-world manipulation tasks, yet learning bimanual robot policies is limited by the scarcity of bimanual robots and datasets. Single-arm robots, however,
X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interaction
Kai Xiong, Hongjie Fang, Lixin Yang et al. · arXiv · May 2026
Effectively handling the interplay between spatial perception and action generation remains a critical bottleneck in robotic manipulation. Existing methods typically treat spatial perception and action execution as decoupled or strictly...
DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair et al. · RSS 2024 · Jul 2024
A 350-hour dataset of diverse real-world robot manipulation across 22 robots in 71 scenes, designed to train scalable and generalist imitation learning policies.
LeRobot: A Library for Real-World Robot Learning
Remi Cadene, Simon Alibert, Alexander Soare et al. · NeurIPS 2024 Workshop · Jun 2024
Hugging Face's LeRobot is an open-source PyTorch framework providing pretrained models, datasets, and training scripts for imitation and reinforcement learning on real robots, lowering the entry barrier to robot learning.