DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation

Siyuan Ma, Boshi Zhang, Yutian Zhang, Qinglian Wu, Jiaqi Zhai, Dong Wei, Qiaojun Yu · N/A · 2026

Summary

Abstract Summary

Mobile manipulation requires a robot to predict how locomotion and arm motion jointly alter future observations and control. Existing world-action models, developed largely for fixed-base platforms, do not explicitly distinguish camera ego-motion from base and arm actions. Here we introduce DECOWAM, a whole-body world-action model that separates these factors through dedicated conditional interfaces. DECOWAM freezes an adapted FastWAM backbone and trains residual adapters, an action-equivalent future bottleneck distilled from privileged observations, adversarially separated base and arm latents, and base-velocity conditioning for video prediction. We further introduce ARMDOG, a real-robot dataset that synchronizes video, whole-body state and action, and language. On a fixed replay protocol, DECOWAM improved both future-video and action prediction over FastWAM, reducing action MSE by 21.7% with 25.95M trainable adaptation parameters. Across 79 closed-loop trials per method, it achieved the highest observed whole-body coordination and base-displacement robustness among the compared systems, while task completion remained comparable to the strongest baseline. These results show that embodiment-aware factorization can support parameter-efficient joint visual prediction and whole-body control under moving viewpoints.

Key Points

Mobile manipulation requires a robot to predict how locomotion and arm motion jointly alter future observations and control.
Existing world-action models, developed largely for fixed-base platforms, do not explicitly distinguish camera ego-motion from base and arm actions.
Here we introduce DECOWAM, a whole-body world-action model that separates these factors through dedicated conditional interfaces.
DECOWAM freezes an adapted FastWAM backbone and trains residual adapters, an action-equivalent future bottleneck distilled from privileged observations, adversarially separated base and arm latents, and base-velocity conditioning for video prediction.
We further introduce ARMDOG, a real-robot dataset that synchronizes video, whole-body state and action, and language.

Method

Let $\ell$ denote a language instruction, $x_{0}$ the current RGB observation, and $s_{0}\in\mathbb{R}^{14}$ the current whole-body state. We learn the conditional joint model where $x_{1:T}$ is a future video and $\mathbf{a}_{1:K}\in\mathbb{R}^{K\times 14}$ is an action chunk. We use $T{=}8$ and $K{=}48$.

Each action has the semantic decomposition The arm and gripper occupy channels $[0{:}7]$, base velocity occupies $[7{:}10]$, and loader padding occupies $[10{:}14]$. Base velocity is both a control target and a source of camera ego-motion.

Method — DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation — Fig. 2: Future-information bottleneck. A privileged teacher observes current and future visual summaries, whereas the causal student observes only the current summary and robot state. Only the student is retained during deployment.

The ARMDOG Dataset

ARMDOG is a model-facing real-robot resource for legged mobile manipulation. It synchronizes moving-camera RGB-D, proprioception, IMU, base state, whole-body commands, and language instructions on a wheeled quadruped robot with 16 leg joints, a 6-DoF arm, and a 1-DoF gripper.

The FastWAM-compatible conversion aligns raw HDF5 streams to 15 Hz and stores each episode as $e_{i}=(V_{i},Q_{i},\ell_{i},\phi(\ell_{i}))$: RGB video, a structured $T_{i}{\times}14$ state/action tensor, instruction text, and a precomputed language embedding.

The ARMDOG Dataset — DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation — Fig. 3: Composition and model-facing structure of ARMDOG . (a) Episode distribution across four task families in the quality-filtered corpus: Bottle Pick&Place (56%), Place Block (39%), Object Pick&Place (4%), and Climb Slope (1%). (b) Corpus scale after timestamp alignment and quality filtering: 1,487 episodes, 343,550 RGB frames, 321.3 minutes, and a 15 Hz frame rate. (c) Model interface: text, current RGB, and robot state form the inputs, while future RGB and a 14-D action are prediction targets. The action vector separates 6-D arm, 1-D gripper, 3-D base-velocity, and 4-D loader-padding channels.

Experiments

The experiments follow an evidence ladder from controlled prediction to physical deployment. We first compare DECOWAM with its FastWAM initialization, then test the structured paths through ablation. Broader reference comparisons establish its position among VLA and WAM systems. Closed-loop trials finally assess task progress, whole-body coordination, and robustness on the physical robot.

Open-loop replay uses a fixed 23-episode box_val__* slice drawn from eight box-manipulation task folders.

Experiments — DECOWAM: Decoupled Whole-Body World-Action Model for Legged Mobile Manipulation — Fig. 5: Representative whole-body coordinated-motion trials on the quadruped–arm platform. The three frames in each row progress from left to right. X-VLA (top) and the original FastWAM (middle) exhibit unintended base motion when a static pose is required and fail to finish the interaction, whereas DECOWAM (bottom) succeeds. The example qualitatively complements the aggregate WBCM-SR results in Table VI-B (b).

Sources and Demonstrations

Figures are reproduced from Ma et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.