General-purpose pretrained models adapted for robotics tasks.
Zuojin Tang, Haoyun Liu, Xinyuan Chang et al. · arXiv · May 2026
Vision-Language-Action (VLA) models map visual observations and language instructions to robot actions through learned l...
Wenxuan Guo, Xiuwei Xu, Yichen Liu et al. · arXiv preprint · May 2026
AwareVLN introduces selective self-aware reasoning for VLN agents, triggering explicit spatial and progress analysis only at uncertain waypoints to improve robustness and explainability.
Jusuk Lee, Seungjae Lee, Jonghun Shin et al. · arXiv preprint · May 2026
Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recog
Xiaosong Jia, Bowen Yang, Zuhao Ge et al. · RSS 2026 · May 2026
Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn...
Qiuxuan Feng, Jiale Yu, Jiaming Liu et al. · arXiv · May 2026
World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics. Current...
Video
Jiye Lee, Yonghun Choi, Jungdam Won · SIGGRAPH 2026 (Journal Track) · Jun 2026
Two-stage framework that enhances noisy multi-human object interaction (MHOI) data by generating plausible hand grasps and refining full-body motion via diffusion-based optimization.
Yajie Li, Bozhou Zhang, Chun Gu et al. · ICML 2026 · May 2026
Video foundation-models models offer a promising imagination mechanism for robot manipulation by predicting long-horizon future observations, but effectively exploiting these imagined futures for action execution remains challenging. Existing approaches...
Zhen Sun, Yongjian Guo, Haoran Sun et al. · arXiv preprint · May 2026
A lightweight runtime verification layer that preemptively prunes unsafe VLA and world-model actions before they reach robot hardware, eliminating collision and droppage failures.
Xinyu Guo, Bin Xie, Wei Chai et al. · arXiv · May 2026
Vision-Language-Action (VLA) models have shown promise for generalizable robot control but struggle when adapting to new...
Qiuyue Wang, Mingsheng Li, Jian Guan et al. · arXiv preprint · May 2026
Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks,
Pengteng Li, Weiyu Guo, He Zhang et al. · arXiv preprint · May 2026
SOMA introduces an explicit spatial memory framework that enables VLA models to track and manipulate objects even after they leave the camera's field of view.
Loizos Hadjiloizou, Rodrigo Pérez-Dattari, Noémie Jaquier · arXiv preprint · May 2026
Formalizes cross-space symmetry compositions in robotics and derives equivariant network architectures that exploit multiple simultaneous symmetries for improved generalization.
Ralf Römer, Maximilian Seeliger, Saida Liu et al. · RSS 2026 — Best Paper Award · Jun 2026
Quantifies epistemic uncertainty in flow-matching VLAs using velocity-field disagreement (VFD) across a small ensemble, enabling failure detection at deployment and sample-efficient active fine-tuning (SAVE).
Junjie Lu, Xinyao Qin, Yuhua Jiang et al. · arXiv · May 2026
Adapting Vision-Language-Action (VLA) models to specific user preferences or task requirements typically requires expens...
Kai Xiong, Hongjie Fang, Lixin Yang et al. · arXiv · May 2026
Effectively handling the interplay between spatial perception and action generation remains a critical bottleneck in robotic manipulation. Existing methods typically treat spatial perception and action execution as decoupled or strictly...
Alexander Khazatsky, Karl Pertsch, Suraj Nair et al. · RSS 2024 · Jul 2024
A 350-hour dataset of diverse real-world robot manipulation across 22 robots in 71 scenes, designed to train scalable and generalist imitation learning policies.
Yecheng Jason Ma, William Liang, Guanzhi Wang et al. · ICLR 2024 (Oral) · May 2024
Eureka uses GPT-4 to write reward functions for RL environments, achieving human-level reward design on 29 tasks and enabling zero-shot sim-to-real transfer on Shadow Hand dexterous manipulation.
Remi Cadene, Simon Alibert, Alexander Soare et al. · NeurIPS 2024 Workshop · Jun 2024
Hugging Face's LeRobot is an open-source PyTorch framework providing pretrained models, datasets, and training scripts for imitation and reinforcement learning on real robots, lowering the entry barrier to robot learning.
Dibya Ghosh, Homer Walke, Karl Pertsch et al. · RSS 2024 · Jul 2024
Octo is a large open-source transformer-based generalist robot policy trained on 800k trajectories, supporting language-conditioned and goal-image-conditioned control across 9 robotic platforms with efficient fine-tuning on consumer GPUs.
Peiqi Liu, Yat Long Lo, Ted Xiao et al. · arXiv preprint · Mar 2024
OK-Robot uses off-the-shelf VLMs (CLIP, OWL-ViT) and LLMs (GPT-4) to perform open-ended object manipulation in unseen homes without any training, achieving 58% success on real-world pick-and-place tasks.
Karl Pertsch, Oliver Groth, Jonas Frey et al. · arXiv preprint · Oct 2024
π0 is a 3.5B-parameter VLA flow model from Physical Intelligence that achieves state-of-the-art general robot manipulation by mixing online RL with high-quality human demonstrations, available as an open-source PyTorch implementation.
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti et al. · arXiv · Jun 2024
A 7B-parameter open-source Vision-Language-Action model pre-trained on 970k real-world robot demonstrations, achieving strong generalization across robots and tasks.
Anthony Brohan, Noah Brown, Justice Carbajal et al. · ICRA 2024 · May 2024
Google DeepMind's VLA model combining a vision-language foundation model with robot action outputs, showing emergent generalization to novel objects, backgrounds, and semantic instructions far beyond training data.
Farama Foundation, Jordan Terry, Mark Towers et al. · JMLR 2023 · Mar 2023
Gymnasium is the maintained successor to OpenAI Gym, providing a standardized API for RL environments with 100+ built-in tasks, vectorized parallel execution, and native support for physics engines like MuJoCo, PyBullet, and IsaacGym.
Jiayuan Gu, Sean Xiang, Stone Tao et al. · ICLR 2023 (Oral) · Jan 2023
ManiSkill is a GPU-parallelized robotics simulation benchmark with 20+ manipulation tasks, PartNet-Mobility assets, and unified observation/action spaces, supporting RL, IL, and VLA training at millions of steps per hour.
Open X-Embodiment Collaboration · arXiv · Oct 2023
The largest collaborative robot learning dataset ever assembled, spanning 22 robots from 21 institutions, enabling training of generalist RT-X policies that exhibit positive cross-embodiment transfer.
Anthony Brohan, Yevgen Chebotar, Chelsea Finn et al. · RSS 2023 · Jul 2023
RT-1 is a 35M-parameter transformer trained on 130K robot demonstrations that generalizes to new tasks, objects, and environments, forming the foundation for Google's RT-2 and RT-X line of VLA models.
Anthony Brohan, Noah Brown, Justice Carbajal et al. · RSS · Jul 2023
A large transformer model trained on 130k episodes of real robot manipulation to output discretized arm-and-gripper actions from RGB images and natural language instructions.
Wei Xu, Yixi Cai, Dongjiao He et al. · IEEE T-RO · 2022
A tightly-coupled LiDAR-inertial odometry system with incremental kd-tree mapping, enabling real-time state estimation for UAVs and mobile robots.
Mohit Shridhar, Lucas Manuelli, Dieter Fox · CoRL 2021 · Nov 2021
CLIPort fuses CLIP's semantic understanding with Transporter Networks' spatial precision to perform language-conditioned manipulation tasks like 'stack the red block on the blue block' with 2D pick-and-place affordances.
Bokui Shen, Fei Xia, Chengshu Li et al. · ICRA 2021 · May 2021
iGibson is a photorealistic, physics-based indoor simulation platform with 15 interactive scenes, 500+ object categories, and GPU-accelerated rendering, enabling RL and VLA training for mobile manipulation and embodied AI.
Mike Lambeta, Po-Wei Chou, Stephen Tian et al. · IEEE RAL · Jul 2020
An affordable, compact, high-resolution vision-based tactile sensor optimized for in-hand manipulation with real-time fingertip deformation sensing.
Josh Tobin, Rachel Fong, Alex Ray et al. · IROS · Sep 2017
Introduced domain randomization for vision-based manipulation: randomize simulation textures, lighting, and camera parameters so that the real world looks like just another variation.
Read these papers in order to build expertise in Foundation Models.
Zuojin Tang, Haoyun Liu et al. · 2026
Wenxuan Guo, Xiuwei Xu et al. · 2026
Jusuk Lee, Seungjae Lee et al. · 2026
Xiaosong Jia, Bowen Yang et al. · 2026
Qiuxuan Feng, Jiale Yu et al. · 2026
Jiye Lee, Yonghun Choi et al. · 2026
…and 27 more papers.
