WT-UMI: Tactile-based Whole-Body Manipulation via Force-Supervised Contact-Aware Planning
Jaehwi Jang, Zhaoyuan Gu, Alfred Cueva, Zimeng Chai, Junjie Sheng, Thong Nguyen, Himank Galundia, Yifan Wu, Huishu Xue, Isaac Legene, Ojas Mediratta, Davin Doan, Andrew Collins, Sarah Sadegh, KyoungMok Kim, Rishita Dhalbisoi, Zun Chen, Ye Zhao · N/A · 2026
Framework
N/A
License
N/A
Stars
N/A
Summary
Whole-body humanoid manipulation of bulky, deformable, and shared-load objects requires distributed contact sensing and explicit force regulation, yet most imitation policies treat contact force only implicitly.
Abstract Summary
Key Points
- Introduces WT-UMI, a wearable whole-body tactile interface for humanoid manipulation.
- Proposes a force-conditioned target-pose correction module to convert human poses into contact-aware robot targets.
- Achieves superior performance on contact-rich tasks including deformable objects and human-humanoid collaboration.
- Demonstrates the importance of explicit force regulation in whole-body manipulation policies.
Methods
Our system architecture is outlined in Fig. 2 . In data collection , WT-UMI records distributed tactile readings and calibrated contact force from a human operator or via teleoperation on a humanoid robot. In pre-trained target-pose correction , a force-conditioned model learns to correct human target poses into robot-executable actions.
In planner training , an action denoiser predicts corrected bimanual pose chunks, while a force head predicts contact-force trajectories.
Experiment Setup
We demonstrate five contact-rich tasks in three categories, with representative deployments shown in Fig. 4 . For deformable object manipulation , yoga ball manipulation ( T1 ) stabilizes and repositions an inflated ball, and pillow reorientation ( T2 ) reorients a soft pillow. For bulky rigid object manipulation , bucket manipulation ( T3 ) repositions a cone-shaped container with diverse loads.
Results
We evaluate WT-UMI across five tasks introduced in Sec. 4 . Representative deployments for T1–T3 are shown in Fig. 4 and used for quantitative evaluation throughout Sec. 5 . For the collaborative tasks T4 and T5, we detail their setup and successful deployments in Appendix 8.5 . This section investigates whether the force head produces accurate, smooth, and temporally aligned force predictions.
To address this, we evaluate our force head (in Sec. 3.3 ) within a ViT-FMT policy.
Sources and Demonstrations
Figures are reproduced from Jang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.
Related Papers
SI-Diff: A Framework for Learning Search and High-Precision Insertion with a Force-Domain Diffusion Policy
Yibo Liu, Stanko Oparnica, Simon Shewchun-Jakaitis et al.
Contact-rich manipulation is fundamental in robotics but poses significant challenges due to uncertainties in relative poses, such as misalignments and small clearances in peg-in-hole tasks.
Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
Junheng Li, Liang Wu, Sergio A. Esteban et al.
In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop π^nMPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills.
Beyond Binary: Sim-to-Real Dexterous Manipulation with Physics-Grounded Contact Representation
Jiahe Pan, Stelian Coros, Jitendra Malik et al.
A primary bottleneck in contact-rich manipulation is the difficulty of collecting real-world data. Sim-to-real reinforcement learning offers a scalable alternative, but the simulation-reality gap prev
Critic Architecture Matters: Dual vs. Unified Critics for Humanoid Loco-Manipulation
Mehmet Turan Yardımcı
Multi-objective reinforcement learning for humanoid robots must coordinate locomotion and manipulation within a single policy.