Robotics Research Hub
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments

Featured

Qiuyue Wang, Mingsheng Li, Jian Guan, Jinhui Ye, Sicheng Xie, Yitao Liu, Junhao Chen, Zhixuan Liang, Jie Zhang, Xintong Hu, Xuhong Huang, Pei Lin, Junyang Lin, Dayiheng Liu, Shuai Bai · · 2026

foundation-models vla manipulation navigation vision-language
PDF arXiv

Framework

N/A

License

N/A

Stars

0

Summary

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks,

Abstract Summary

Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks, environments, and robot embodiments. In this work, we study whether heterogeneous embodied decision-making problems can be unified within a single vision-language-action model. We present Qwen-VLA, a unified embodied foundation model that extends Qwen's vision-language modeling stack from perception, understanding, and reasoning to continuous action and trajectory generation through a DiT-based action decoder. Qwen-VLA is trained with a large-scale joint pretraining recipe over diverse data sources, including robotics manipulation trajectories, human egocentric demonstrations, synthetic simulation data, vision-and-language navigation data, trajectory-centric supervision, and auxiliary vision-language data. To support multiple robot platforms, we introduce embodiment-aware prompt conditioning, where robot-specific textual descriptions specify the current embodiment and control convention. We further cast manipulation, navigation, and trajectory prediction into a unified action-and-trajectory prediction framework, enabling transferable visual grounding, spatial reasoning, and continuous action generation across robot morphologies, task families, and environments. Experiments on manipulation, navigation, and trajectory-centric benchmarks show consistent multi-task performance and out-of-distribution generalization under variations in scene layout, background, lighting, object configuration, and robot embodiment. Qwen-VLA-Instruct achieves 97.9% on LIBERO, 73.7% on Simpler-WidowX, 86.1%/87.2% on RoboTwin-Easy/Hard, 69.0% OSR on R2R, 59.6% SR on RxR, 76.9% average OOD success in real-world ALOHA experiments, and 26.6% zero-shot success on DOMINO dynamic manipulation.

Key Points

  • Unified VLA foundation model across manipulation, navigation, and trajectory prediction.
  • DiT-based action decoder extending Qwen VL stack to continuous action generation.
  • Embodiment-aware prompt conditioning supports multiple robot platforms.
  • Trained on diverse data including human egocentric, synthetic, and navigation data.
  • Achieves strong results across LIBERO, Simpler, RoboTwin, R2R, RxR, and real-world ALOHA.

Unified Embodied Model

We study a broad family of embodied decision-making tasks, including robot manipulation, vision-and-language navigation, trajectory prediction, and human egocentric action modeling.

We propose to address these tasks within a unified embodied model, motivated by their shared computational structure: despite differences in output format and evaluation protocol, all tasks require an agent to ground language in visual observations, reason over spatial and temporal context, and predict future actions or trajectories.

Unified Embodied Model — Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Figure 2: Training recipe of Qwen-VLA. Stage I (T2A) trains the DiT action decoder to reconstruct actions from text alone, building a structured action prior without visual input. Stage II (CPT) unfreezes both modules to ground this prior in visual observations. Stage III (SFT) branches into multi-task and real-robot tracks, and Stage IV (RL) optimizes closed-loop task success via environment rewards.

Large-Scale Joint Pretraining

A usable VLA model requires co-training a cognitive backbone and a motor decoder, a division of labor analogous to the complementary roles of the cerebrum and cerebellum in biological motor control. In practice, however, these two modules enter training in deeply asymmetric states: the VLM backbone is already strongly pretrained, while the DiT action decoder is randomly initialized.

Naive joint training from this starting point is inefficient and unstable.

Large-Scale Joint Pretraining — Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Figure 3: Examples of data generated through RoboInF . The top row shows a short-horizon task, “Place the two green staplers side by side,” which consists of a compact sequence of reaching, grasping, transporting, and placing. The bottom row shows a long-horizon task, “Group the drinks together and leave the cleaning sponge by itself,” which requires multiple object manipulations and can be decomposed into subtask segments such as picking and placing each drink.

Post-Training

The large-scale pretraining of T2A and CPT produces Qwen-VLA-Base , a generalist vision-language-action model that exhibits broad cross-task and cross-embodiment generalization. Although this broad coverage equips the model with versatile knowledge, it does not yet yield the precision needed for reliable closed-loop control on specific downstream tasks.

Post-Training — Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Figure 4: Overview of real-world evaluation tasks on the ALOHA bimanual platform.

Experiments

We conduct extensive experiments to evaluate the performance of Qwen-VLA in both simulation and real-world settings, covering two core embodied AI domains: robotic manipulation and visual navigation.

We evaluate two model variants throughout: Qwen-VLA-Base , trained with large-scale pretraining on diverse embodied data, and Qwen-VLA-Instruct , further fine-tuned with instruction-following data collected across diverse simulated environments and tasks, capable of executing fine-grained manipulation and navigation with greater precision and relia...

Experiments — Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Figure 5: Qualitative out-of-distribution generalization of Qwen-VLA-Base on the ALOHA dual-arm robot. Top-left: color-conditioned grasping of green, blue, red, and yellow balls. Top-right: upper two panels show grasping of novel objects (green broccoli, toy duck); lower two panels show a compositional “clean up the table” task with sequential pick-and-place into a bin (blue umbrella, toy duck, bottled yogurt). Bottom-left: interaction with completely unseen objects (sunglasses, plush doll, toy duck). Bottom-right: atomic action transfer through uncapping a pen and placing the cap, robust to unseen yellow backgrounds.

Sources and Demonstrations

Figures are reproduced from Wang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers