OpenVLA: An Open-Source Vision-Language-Action Model

OpenVLA: An Open-Source Vision-Language-Action Model

Featured

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Erin Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Peng Xu, Chelsea Finn · Stanford, UC Berkeley, Toyota Research Institute, Google DeepMind, MIT · 2024

Framework

PyTorch

License

Apache-2.0

Stars

6,260

Summary

A 7B-parameter open-source Vision-Language-Action model pre-trained on 970k real-world robot demonstrations, achieving strong generalization across robots and tasks.

Abstract Summary

OpenVLA is one of the first truly open-source Vision-Language-Action (VLA) models designed for generalist robot manipulation. Unlike earlier closed systems such as RT-2, OpenVLA makes both its model weights and training pipeline publicly available, enabling researchers to fine-tune and deploy the model on their own hardware. At its core, OpenVLA is built on top of a 7-billion-parameter Llama 2 language backbone, augmented with a vision encoder (DINOv2 + SigLIP) to process camera observations. The model is trained on the Open X-Embodiment dataset, a massive collection of 970,000 real-world robot trajectories spanning 22 different robot embodiments and hundreds of diverse manipulation tasks. What makes OpenVLA particularly exciting is its strong generalization capability. Because the pre-training dataset covers so many different robots, grippers, and environments, the model learns a broad prior over "how to manipulate objects" rather than overfitting to a single platform. When fine-tuned on a new robot with just a few dozen demonstrations, OpenVLA often outperforms closed commercial models and specialist policies trained from scratch. The model predicts continuous action tokens (end-effector poses and gripper states) by quantizing actions into discrete bins and generating them autoregressively, similar to how a language model generates text. From a systems perspective, OpenVLA is designed for practical deployment. The authors provide a full fine-tuning stack using LoRA adapters, allowing researchers to adapt the model to new robot setups on a single consumer GPU in under an hour. The model also supports language-conditioned task specification: you can prompt it with natural language instructions like "put the red cup on the left plate," and it will generate the corresponding control sequence. This bridges the gap between high-level human intent and low-level motor control, a key challenge in household robotics. The paper provides extensive benchmarking across multiple real robot platforms, including the WidowX and Google Robot. Results show that OpenVLA achieves state-of-the-art performance among open models and even surpasses RT-2-X in several few-shot transfer scenarios. The open release includes model checkpoints, training code, and evaluation scripts, making it an excellent starting point for labs entering the VLA space.

Key Points

  • First fully open-source 7B VLA model with publicly available weights and training code.
  • Pre-trained on 970k real robot trajectories from the Open X-Embodiment dataset (22 robots).
  • Uses Llama 2 + DINOv2/SigLIP vision encoder; actions quantized into discrete tokens.
  • Strong few-shot transfer: fine-tune on ~10-100 demos with LoRA for a new robot.
  • Supports natural language task specification and multi-step long-horizon tasks.

Additional Notes

Fine-tuning Tips

  • Use the official LoRA config for fast adaptation; full fine-tuning requires significant compute.
  • Action quantization bin count is a key hyperparameter when transferring to robots with different action spaces.
  • The model expects image resolution of 224x224; preprocessing should match the training pipeline exactly.
  • RT-2 (Brohan et al., 2023)
  • Octo (Team et al., 2024)
  • Open X-Embodiment (Padalkar et al., 2023)
Share

Related Papers

DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation

DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation

Jusuk Lee, Seungjae Lee, Jonghun Shin et al. · arXiv preprint · May 2026

Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recog

foundation-models perception manipulation representation-learning vla
PDF Intermediate
No code repo Code updated: May 2026