Robotics Research Hub
SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models

SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models

Mousumi Das, Aditeya Prajapati, Abrar Anwar, Jesse Thomason · N/A · 2026

manipulation vision vla reinforcement-learning
PDF arXiv Code (GitHub)

Framework

N/A

License

N/A

Stars

N/A

Summary

Robot manipulation systems using Vision-Language-Action (VLA) model backbones typically use just one VLA for task execution. However, individual VLAs do not perform well across different task states and environments.

Abstract Summary

Robot manipulation systems using Vision-Language-Action (VLA) model backbones typically use just one VLA for task execution. However, individual VLAs do not perform well across different task states and environments. We introduce a framework for dynamically composing multiple VLA policies during execution: StepWise Action Policy Routing (SWAP). SWAP formulates policy routing as an offline reinforcement learning problem, learning a routing critic that selects the most appropriate policy at each decision step given the current observation. SWAP enables robots to select new policies to execute online rather than committing to a single policy for the duration of an episode. We evaluate SWAP on both real-world DROID manipulation tasks and LIBERO simulation experiments. SWAP improves over fixed-policy execution and routing baselines, giving absolute improvements in real-world task success up to 33% while reducing successful trajectory robot action step length by 28.3%.

Key Points

  • Robot manipulation systems using Vision-Language-Action (VLA) model backbones typically use just one VLA for task execution.
  • However, individual VLAs do not perform well across different task states and environments
  • We introduce a framework for dynamically composing multiple VLA policies during execution: StepWise Action Policy Routing (SWAP).
  • SWAP formulates policy routing as an offline reinforcement learning problem, learning a routing critic that selects the most appropriate policy at each decision step given the current observation.
  • SWAP enables robots to select new policies to execute online rather than committing to a single policy for the duration of an episode.

Robot Policy Routing

While modern VLA policies can exhibit strong generalization, no single policy is uniformly optimal across all task states, environments, or execution regimes. A policy that performs well during coarse free-space motion may fail during contact-rich manipulation, while another may excel at precise object interaction yet behave conservatively during exploration or recovery.

Robot Policy Routing — SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models
Fig. 2 : Overview of SWAP. (a) We construct an offline routing dataset by collecting trajectories from a set of candidate robot policies and chunking them into routing transitions labeled with sparse rewards and termination signals. (b) SWAP trains a routing critic offline using IQL, where visual observations and task instructions are encoded with a frozen vision-language model and combined with a candidate policy embedding to predict policy-specific Q-values. (c) SWAP evaluates all candidate policies under the current observation, selects the highest-valued policy, executes it in the environment for a short horizon, and repeats this process online throughout task execution.

SWAP: Stepwise Action Policy Routing

We instantiate the policy routing framework as S tep- W ise A ction P olicy Routing (SWAP), an offline reinforcement learning approach for selecting among candidate robot policies during robot action rollouts (Figure 2 ). SWAP learns a routing critic that estimates which policy is most likely to lead to successful task completion given the robot’s current execution state.

SWAP: Stepwise Action Policy Routing — SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models
Fig. 3 : Evaluation environments. We evaluate SWAP on both real-world DROID [ 1 ] robot manipulation tasks and LIBERO [ 48 ] simulation tasks. For simulation, we use the spatial subset of LIBERO-Plus [ 18 ] , which includes challenging out-of-distribution task variations.

Experiments and Results

We evaluate SWAP in both real-world and simulated manipulation settings to assess whether learned stepwise policy routing improves robot manipulation performance over fixed policy execution. In each setting, SWAP is trained using offline interaction data collected from the candidate policies. We compare SWAP against two classes of baselines.

First, we evaluate each candidate policy independently, corresponding to the standard setting where a single policy is executed for the full episode without routing.

Experiments and Results — SWAP: Stepwise Action Policy Routing for Vision-Language-Action Models
Fig. 4 : Q-values and policy routing on the DROID marker-in-bowl task. SWAP scores $\pi_{0}$-FAST, $\pi_{0}$, and $\pi_{0.5}$ based on the predicted Q-values of each policy throughout task execution. The images highlight key routing decisions in one SWAP rollout. The plot shows the corresponding Q-values over time, as well as the policy selected by SWAP at each decision step. Initially, SWAP favors $\pi_{0}$-FAST, which takes larger steps during free-space movement. As the robot approaches the object, it switches to $\pi_{0.5}$ for the more precise grasp. After the marker is accidentally dropped, the Q-values collapse and SWAP again selects $\pi_{0.5}$ to recover and reattempt the grasp. Once the marker is secured, SWAP switches back to $\pi_{0}$-FAST for faster motion to the goal.

Sources and Demonstrations

Figures are reproduced from Das et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
arXiv preprint

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Haoran Chen, Jingtian Ji, Samuel Wheeler et al.

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.

manipulation vision vla reinforcement-learning control benchmark
PDF Intermediate