HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models
Qiuxuan Feng, Jiale Yu, Jiaming Liu, Yueru Jia, Zhuangzhe Wu · · 2026
Framework
N/A
License
N/A
Stars
7
Summary
World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics. Current WAMs generally follow two paradigms: generalizable but imprecise models learned from diverse data, or precise but specialized models fitted to specific tasks.
Abstract Summary
Key Points
- Proposes HarmoWAM, a novel approach for foundation models in robotics.
- Addresses key limitations in existing methods through innovative architecture design.
- Demonstrates strong empirical results on standard benchmarks and real-world evaluations.
- Introduces novel training methodology that improves generalization and sample efficiency.
- Provides comprehensive analysis of failure modes and ablation studies.
Methods
Action Prediction Problem Formulation. We formulate robotic manipulation as a probabilistic sequential decision-making problem Black et al. (2024) .
At each timestep $t$, conditioned on a natural language instruction $\mathbf{l}_{t}$ and a visual observation $\mathbf{I}_{t}\in\mathbb{R}^{H\times W\times 3}$, the policy $\pi_{\theta}$ predicts a future action sequence of length $H$, defined as $\mathbf{a}_{t+1:t+H}\sim\pi_{\theta}(\cdot\mid\mathbf{I}_{t},\mathbf{l}_{t})$.
Experiments
Section 4.1 introduces the experimental setup. Section 4.2 reports results on real-world in-domain (ID) tasks, while Section 4.3 evaluates performance on out-of-domain (OOD) scenarios to assess generalization. Section 4.4 presents ablation studies of key components. Data Collection.
We evaluate HarmoWAM on real-world manipulation tasks using dual-arm Franka Research 3 robots, each equipped with three Intel RealSense cameras (one third-person and two wrist-mounted views).
Sources and Demonstrations
Figures are reproduced from Feng et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
Jusuk Lee, Seungjae Lee, Jonghun Shin et al.
Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recog
EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
Christen Millerdurai, Shaoxiang Wang, Yaxu Xie et al.
Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must remain compact and unobtrusive.
GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
Xiaosong Jia, Bowen Yang, Zuhao Ge et al.
Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn task-relevant features.
Video
MOCHI: Motion Enhancement of Collaborative Human-object Interactions
Jiye Lee, Yonghun Choi, Jungdam Won
Two-stage framework that enhances noisy multi-human object interaction (MHOI) data by generating plausible hand grasps and refining full-body motion via diffusion-based optimization.