Robotics Research Hub
OpenVLA: An Open-Source Vision-Language-Action Model

OpenVLA: An Open-Source Vision-Language-Action Model

Featured

Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Erin Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Peng Xu, Chelsea Finn · Stanford, UC Berkeley, Toyota Research Institute, Google DeepMind, MIT · 2024

vla llm-robotics foundation-models manipulation
PDF arXiv Code (6260★)

Framework

PyTorch

License

Apache-2.0

Stars

6,260

Summary

A 7B-parameter open-source Vision-Language-Action model pre-trained on 970k real-world robot demonstrations, achieving strong generalization across robots and tasks.

Abstract Summary

OpenVLA is one of the first truly open-source Vision-Language-Action (VLA) models designed for generalist robot manipulation. Unlike earlier closed systems such as RT-2, OpenVLA makes both its model weights and training pipeline publicly available, enabling researchers to fine-tune and deploy the model on their own hardware. At its core, OpenVLA is built on top of a 7-billion-parameter Llama 2 language backbone, augmented with a vision encoder (DINOv2 + SigLIP) to process camera observations. The model is trained on the Open X-Embodiment dataset, a massive collection of 970,000 real-world robot trajectories spanning 22 different robot embodiments and hundreds of diverse manipulation tasks. What makes OpenVLA particularly exciting is its strong generalization capability. Because the pre-training dataset covers so many different robots, grippers, and environments, the model learns a broad prior over "how to manipulate objects" rather than overfitting to a single platform. When fine-tuned on a new robot with just a few dozen demonstrations, OpenVLA often outperforms closed commercial models and specialist policies trained from scratch. The model predicts continuous action tokens (end-effector poses and gripper states) by quantizing actions into discrete bins and generating them autoregressively, similar to how a language model generates text. From a systems perspective, OpenVLA is designed for practical deployment. The authors provide a full fine-tuning stack using LoRA adapters, allowing researchers to adapt the model to new robot setups on a single consumer GPU in under an hour. The model also supports language-conditioned task specification: you can prompt it with natural language instructions like "put the red cup on the left plate," and it will generate the corresponding control sequence. This bridges the gap between high-level human intent and low-level motor control, a key challenge in household robotics. The paper provides extensive benchmarking across multiple real robot platforms, including the WidowX and Google Robot. Results show that OpenVLA achieves state-of-the-art performance among open models and even surpasses RT-2-X in several few-shot transfer scenarios. The open release includes model checkpoints, training code, and evaluation scripts, making it an excellent starting point for labs entering the VLA space.

Key Points

  • First fully open-source 7B VLA model with publicly available weights and training code.
  • Pre-trained on 970k real robot trajectories from the Open X-Embodiment dataset (22 robots).
  • Uses Llama 2 + DINOv2/SigLIP vision encoder; actions quantized into discrete tokens.
  • Strong few-shot transfer: fine-tune on ~10-100 demos with LoRA for a new robot.
  • Supports natural language task specification and multi-step long-horizon tasks.

The OpenVLA Model

We introduce the OpenVLA model, a 7B-parameter vision-language-action model (VLA) trained on 970k robot demonstrations from the Open X-Embodiment dataset [ 1 ] . There are many, largely unexplored, questions around best practices for developing VLA models, e.g., what are the best model backbones, datasets, and hyperparameters to use for training.

Below, we detail our approach for developing OpenVLA and summarize our key learnings.

The OpenVLA Model — OpenVLA: An Open-Source Vision-Language-Action Model
Figure 2: OpenVLA model architecture. Given an image observation and a language instruction, the model predicts 7-dimensional robot control actions. The architecture consists of three key components: (1) a vision encoder that concatenates Dino V2 [ 25 ] and SigLIP [ 79 ] features, (2) a projector that maps visual features to the language embedding space, and (3) the LLM backbone , a Llama 2 7B-parameter large language model [ 10 ] .

The OpenVLA Codebase

Along with our model, we release the OpenVLA codebase, a modular PyTorch codebase for training VLA models (see https://openvla.github.io ).

It scales from fine-tuning VLAs on individual GPUs to training billion-parameter VLAs on multi-node GPU clusters, and supports modern techniques for large transformer model training such as automatic mixed precision (AMP, PyTorch [75] ), FlashAttention [ 76 ] , and fully sharded data parallelism (FSDP, Zhao et al. [77] ).

The OpenVLA Codebase — OpenVLA: An Open-Source Vision-Language-Action Model
Figure 3: BridgeData V2 WidowX robot evaluation tasks and results. We evaluate OpenVLA and prior state-of-the-art generalist robot policies on a comprehensive suite of tasks covering several axes of generalization, as well as tasks that specifically assess language conditioning ability. OpenVLA achieves highest overall performance and even outperforms closed-source model RT-2-X in all categories except for semantic generalization. Average success rates $\pm$ StdErr are computed across 170 total rollouts per approach. See Table 4 for detailed results.

Experiments

The goal of our experimental evaluations is to test OpenVLA’s ability to serve as a powerful multi-robot control policy out of the box, as well as be a good initialization for fine-tuning to new robot tasks.

Experiments — OpenVLA: An Open-Source Vision-Language-Action Model
Figure 4: Google robot evaluation results. We evaluate generalist robot policies on in-distribution and out-of-distribution (OOD) tasks on the mobile manipulator used in RT-1 and RT-2 evaluations [ 2 , 7 ] . We find that OpenVLA and RT-2-X attain comparable performance and significantly outperform RT-1-X and Octo overall. Average success rates $\pm$ StdErr are computed across 60 total rollouts per approach. See Table 6 for detailed results.

Sources and Demonstrations

Figures are reproduced from Kim et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers