FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation

Shiyuan Yang, Borong Zhang, Jizheng Zhang, Zhijia Tao, Junfei Guo, Donglai Ran, Xu Bian, Qingbiao Li · N/A · 2026

Summary

We present FabriVLA, a lightweight Vision-Language-Action model for Precise Multi-Task Manipulation.

Abstract Summary

We present FabriVLA, a lightweight Vision-Language-Action model for Precise Multi-Task Manipulation. FabriVLA combines an InternVL3.5 vision-language backbone with a flow-matching action head featuring gated self-attention across action tokens and shallow VLM layer fusion for enriched spatial context. The model is trained via single stage joint optimization from a pretrained VLM and randomly initialized action head. On the Meta-World MT50 benchmark spanning 50 diverse manipulation tasks, FabriVLA achieves a tier-average success rate of 90.0%, demonstrating that a compact VLA built on a 1B scale VLM can achieve strong performance without relying on multi billion parameter VLA backbones.

Key Points

We present FabriVLA, a lightweight Vision-Language-Action model for Precise Multi-Task Manipulation
FabriVLA combines an InternVL3.5 vision-language backbone with a flow-matching action head featuring gated self-attention across action tokens and shallow VLM layer fusion for enriched spatial context.
The model is trained via single stage joint optimization from a pretrained VLM and randomly initialized action head.
On the Meta-World MT50 benchmark spanning 50 diverse manipulation tasks, FabriVLA achieves a tier-average success rate of 90.0%, demonstrating that a compact VLA built on a 1B scale VLM can achieve strong performance without relying on multi billion parameter VLA backbones.

Model Architecture

FabriVLA consists of three components: (1) an InternVL3.5 vision-language backbone, (2) state and action encoders, and (3) a flow-matching action head. Figure 2 illustrates the overall architecture. We adopt InternVL3.5-1B ( Wang et al. 2025 ) as the VLM backbone. Input images are resized to $448\times 448$ and processed through the vision encoder, which produces a sequence of visual tokens.

A task prompt (e.g., “Dunk the basketball into the basket”) is tokenized via the language tokenizer and concatenated with the visual tokens.

Model Architecture — FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation — Figure 2: Overview of FabriVLA. RGB observations and the task instruction are encoded by an InternVL3.5-1B vision-language backbone (14 retained layers) into fused context tokens. A shallow VLM layer (layer 6) is additionally extracted and fused with the deep features via a learned linear projection to enrich spatial detail. A flow-matching action head composed of a stack of $L{=}8$ gated cross-attention blocks denoises an action chunk conditioned on these tokens, the robot state, and the flow timestep. The entire model is trained jointly from a pretrained VLM and randomly initialized action head.

Training Procedure

FabriVLA is trained in a single stage on the openly released Evo-1 Meta-World demonstration dataset ( Lin et al. 2026c ) , which contains 50 trajectories per MT50 task and 2,500 trajectories in total. The action head is randomly initialized; the InternVL3.5 backbone is loaded from the publicly released pretrained checkpoint.

The entire model, both the VLM backbone and the action head is jointly optimized from the start, with all parameters unfrozen throughout training.

Training Procedure — FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation — Figure 3: Design space of optional modules in the FabriVLA action head. All three variants share a common base head—action encoder $\rightarrow$ $L{=}8$ cross-attention blocks $\rightarrow$ LayerNorm $\rightarrow$ sequence-pooled MLP head $\rightarrow$ velocity—and each adds a single zero-initialized gated residual module, so at initialization (gate $=0$) every variant reduces exactly to the base head; they differ only in where temporal or per-step information is injected. (a) Gated self-attention mixes the 50 action tokens inside every block before cross-attention (the learnable gate $g$ of Section 2 ). (b) A token residual head adds a per-step linear map $\mathbb{R}^{1024}\!\rightarrow\!\mathbb{R}^{24}$ in parallel with the pooled MLP head. (c) A depthwise temporal convolution ($k{=}3$) smooths adjacent steps along the horizon after the block stack. The released FabriVLA enables variant (a) only.

Experiments

We evaluate FabriVLA on the Meta-World MT50 benchmark ( Yu et al. 2020 ) , which comprises 50 distinct robotic manipulation tasks grouped into four difficulty tiers: easy, medium, hard, and very hard. Tasks range from simple reaching and pushing to complex articulated object manipulation (e.g., door opening, peg insertion, nut assembly).

Each task is evaluated over 10 episodes with randomized initial states using environment seed 4042. An episode is considered successful if the task goal is achieved within a 400 step horizon.

Experiments — FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation — Figure 4: Meta-World MT50 success rate versus training step. Colored lines are the four difficulty tiers; the solid black and dashed gray lines are the tier-average and overall episode-level success rates. The 93k checkpoint corresponds to the selected result in Table 1 .

Sources and Demonstrations

Figures are reproduced from Yang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.