12 papers with open-source code, published in 2024.
Karl Pertsch, Oliver Groth, Jonas Frey et al.
π0 is a 3.5B-parameter VLA flow model from Physical Intelligence that achieves state-of-the-art general robot manipulation by mixing online RL with high-quality human demonstrations, available as an open-source PyTorch implementation.
Alexander Khazatsky, Karl Pertsch, Suraj Nair et al.
A 350-hour dataset of diverse real-world robot manipulation across 22 robots in 71 scenes, designed to train scalable and generalist imitation learning policies.
Dibya Ghosh, Homer Walke, Karl Pertsch et al.
Octo is a large open-source transformer-based generalist robot policy trained on 800k trajectories, supporting language-conditioned and goal-image-conditioned control across 9 robotic platforms with efficient fine-tuning on consumer GPUs.
Xuanlin Li, Kyle Hsu, Jiayuan Gu et al.
A benchmark environment for evaluating real-world robot manipulation policies in simulation using real-world video prompts, enabling zero-shot sim-to-real transfer evaluation without physical hardware.
Remi Cadene, Simon Alibert, Alexander Soare et al.
Hugging Face's LeRobot is an open-source PyTorch framework providing pretrained models, datasets, and training scripts for imitation and reinforcement learning on real robots, lowering the entry barrier to robot learning.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti et al.
A 7B-parameter open-source Vision-Language-Action model pre-trained on 970k real-world robot demonstrations, achieving strong generalization across robots and tasks.
Yecheng Jason Ma, William Liang, Guanzhi Wang et al.
Eureka uses GPT-4 to write reward functions for RL environments, achieving human-level reward design on 29 tasks and enabling zero-shot sim-to-real transfer on Shadow Hand dexterous manipulation.
Anthony Brohan, Noah Brown, Justice Carbajal et al.
Google DeepMind's VLA model combining a vision-language foundation model with robot action outputs, showing emergent generalization to novel objects, backgrounds, and semantic instructions far beyond training data.
Nikita Rudin, Fabian Jenelten, Timon Homberger et al.
Unitree Go2 uses cross-modal vision + proprioception transformers trained in Isaac Gym to achieve agile navigation over rough terrain with minimal sim-to-real gap.
Peiqi Liu, Yat Long Lo, Ted Xiao et al.
OK-Robot uses off-the-shelf VLMs (CLIP, OWL-ViT) and LLMs (GPT-4) to perform open-ended object manipulation in unseen homes without any training, achieving 58% success on real-world pick-and-place tasks.
Zachary Kousik, Shanpu Fang, Rohan Thakker et al.
An open-source 3D-printed humanoid robot platform designed for reinforcement-learning based locomotion research with affordable hardware.
Wang J., Xiao B., Wang J., Heider P., Li H., Zhang Y.
An Isaac-Gym-based training framework for humanoid locomotion that enables zero-shot sim-to-real transfer on XBot-L/XBot-S robots via domain randomization.
