Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
Haoran Chen, Jingtian Ji, Samuel Wheeler, Kaylene Caswell Stocking, Matthew Walter · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However,...
Abstract Summary
Key Points
- Autoregressive action-token policies such as vision-language-action models require action tokeniz...
- Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective
- However, as we show through extensive analysis, sufficiently accurate action reconstruction is on...
- It is also critical that the policy is able to predict the right tokens for new observations, and...
- These properties are downstream of tokenizer training and are not directly incentivized by a reco...
Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
|Authors: Haoran Chen, Jingtian Ji, Samuel Wheeler, Kaylene Caswell Stocking, Matthew Walter
|Venue: arXiv preprint | Year: 2026
|arXiv: 2610.09170
Abstract
Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective. However, as we show through extensive analysis, sufficiently accurate action reconstruction is only one part of what makes a downstream robot policy successful. It is also critical that the policy is able to predict the right tokens for new observations, and that unseen policy token predictions still decode into reasonable actions. These properties are downstream of tokenizer training and are not directly incentivized by a reconstruction objective alone. In this work, we introduce Predictable and Robust Action Tokenization (ProAct), a tokenizer training method that strategically augments reconstruction with the goal of improving downstream predictability and robustness. ProAct is policy-agnostic and uses only action datasets for training. Across the Robomimic, LIBERO, and RoboTwin benchmarks and a diverse set of tokenizer architectures, ProAct improves rollout success by an average of 11.3 percentage points. These improvements also translate to vision-language-action policies and real-world robotic manipulation, yielding average gains of 21.8 and 36.7 percentage points, respectively. These results suggest that effective action tokenization should be designed as a policy interface that balances fidelity, predictability, and robustness, rather than as a reconstruction problem alone.
Key Contributions
- Autoregressive action-token policies such as vision-language-action models require action tokeniz…
- Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective
- However, as we show through extensive analysis, sufficiently accurate action reconstruction is on…
- It is also critical that the policy is able to predict the right tokens for new observations, and…
- These properties are downstream of tokenizer training and are not directly incentivized by a reco…
Topics
- manipulation
- vision
- vla
- reinforcement-learning
- control
- benchmark
Code & Data
No code repository linked in paper metadata.
BibTeX
@article{Chen2026_261009170,
title = {Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?},
author = {Haoran Chen and Jingtian Ji and Samuel Wheeler and Kaylene Caswell Stocking and Matthew Walter},
year = {2026},
eprint = {2610.09170},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2610.09170}
}
Related Papers
A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.
Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions. Most state-of-the-art models predict actions in the end-effector pose space as ...
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
Zhaoda Du, Michael Bowman, Xiaoli Zhang
Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping. We present a capability-aware shared-control framework in which a vision-language model (...
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
Yunchao Yao, Zhuxiu Xu, Tianqi Zhang et al.
Building general-purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments. However, existing benchmarks remain limited in task and data diversity, embod...
DUET: Decoupled Upper- and Lower-Body Execution for Task-Agnostic Humanoid Loco-Manipulation
Kapalik Khanal, Biswash Khatiwada, Stephen Afrifa et al.
Changing a humanoid robot's manipulation policy should not require retraining the controller that maintains the robot's balance. This study introduces DUET, a decoupled execution framework that enables locomotion and manipulation controllers to operate simultaneously by combining their commands o...