GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
Xiaosong Jia, Bowen Yang, Zuhao Ge, Xian Nie, Yuchen Zhou, Cunxin Fan, Yufeng Li, Yilin Chai, Chao Jing, Zijian Liang, Qingwen Bu, Haidong Cao, Chao Wu, Qifeng Li, Zhenjie Yang, Chenhe Zhang, Hongyang Li, Zuxuan Wu, Junchi Yan, Yu-Gang Jiang · · 2026
Framework
N/A
License
N/A
Stars
0
Summary
Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn task-relevant features.
Abstract Summary
Key Points
- Introduces GuidedVLA, advancing vla research.
- Detailed evaluation demonstrates effectiveness on benchmarks and real-world scenarios.
- Provides insights into methodology and architectural choices.
Method
Recent representative Vision-Language-Action (VLA) models [ 8 , 45 , 104 ] extend Vision-Language Models (VLMs) by introducing action tokens $\bm{a}$ alongside vision tokens $\bm{v}$ and language tokens $\bm{l}$. The action generation process is trained to regress robot trajectories by denoising $\bm{a}$ conditioned on $(\bm{v},\bm{l})$.
Although VLMs provide rich semantic features in $(\bm{v},\bm{l})$, the action tokens $\bm{a}$ do not inherently learn to extract task-critical information.
Experiments
LIBERO-Plus [ 28 ] is a robustness-oriented benchmark built upon LIBERO [ 59 ] . It is designed to evaluate generalist manipulation policies under distribution shifts .
It introduces perturbations along seven dimensions: camera viewpoint, robot initial state, language variation, lighting condition, background texture, sensor noise, and object layout to expose failure modes under generalization scenario beyond in-domain evaluation. We compare with state-of-the-art baselines in Table I .
Analysis
Do VLAs under-utilize vision-language representations in action decoding process, and can explicit factor guidance close this gap? (Section V-B ) Does our proposed GuidedVLA improve baseline performance under both in-distribution and out-of-distribution evaluations? (Section V-A ) Which factors (object, skill, geometry) matter most for which task types?
(Section V-A ) Does our attention head specialization indeed lead to learning decoupled features? (Section V-C ) How different architectural choices for guidance influence performance? (Section...
Sources and Demonstrations
Figures are reproduced from Jia et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
Jusuk Lee, Seungjae Lee, Jonghun Shin et al.
Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recog
From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulation
Yajie Li, Bozhou Zhang, Chun Gu et al.
Video foundation-models models offer a promising imagination mechanism for robot manipulation by predicting long-horizon future observations, but effectively exploiting these imagined futures for action execution remains challenging.
Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodiments
Qiuyue Wang, Mingsheng Li, Jian Guan et al.
Embodied intelligence is often studied through specialized models for individual tasks such as manipulation or navigation, resulting in fragmented capabilities and limited generalization across tasks,
Spatial Memory for Out-of-Vision Manipulation in Vision-Language-Action
Pengteng Li, Weiyu Guo, He Zhang et al.
SOMA introduces an explicit spatial memory framework that enables VLA models to track and manipulate objects even after they leave the camera's field of view.