HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation

At a glance

Problem: collecting humanoid demonstrations normally ties up the target robot. Source
Idea: predict sparse body keypoints, retarget them, then track whole-body references. Source
Evidence: physical G1 demonstrations cover manipulation, throwing, bending and walking. Source

Evaluation scope The collection-throughput comparison uses two operators, including a novice given ten minutes of instruction. Source

Collection hardware: body trackers and camera-equipped grippers capture the human demonstration without a robot in the loop. Original figure

How it works

Record synchronized wrist images, tracked body points and gripper openings using the portable VR interface. Source
Predict keypoint trajectories with a diffusion policy and convert them into robot-native references. Source
Track those references with the low-level whole-body controller. Source

Three-stage control: keypoint prediction, spatial retargeting and whole-body tracking have distinct responsibilities. Original figure

Evidence & results

In ten-minute bimanual collection sessions, the novice gathered 55 valid demonstrations versus 25 with TWIST2; the experienced operator gathered 55 versus 30. Evidence
Adding knee keypoints improves lower-body coordination for waste disposal and walking delivery compared with the default five-keypoint configuration. Evidence

G1 task sequences and ablations cover tabletop placement, two-arm collection and throwing. The plots compare retargeting and latency-matching variants. Original figure

Limitations & boundaries

The throughput study has only two operators. Its task-specific speedups should not be treated as a general productivity guarantee. Source
Five keypoints underconstrain demanding lower-body motion; the augmented configuration adds knee tracking. Source

Takeaway

Robot-free collection is useful only when retargeting and whole-body execution preserve the demonstrated spatial relationships. Source

Resources

Full text ↗PDF ↗Repository ↗

Figures: the paper's authors. Editorial source and image review: 2026-10-21. Findings are author-reported, not independently reproduced by Robotics Research Hub.

Citation

@article{Wang2026_260627239v1, title={HumanoidUMI: Bridging Robot-Free Demonstrations and Humanoid Whole-Body Manipulation}, author={Hongwu Wang and Chenhao Yu and Youhao Hu and Jiachen Zhang and Yuanyuan Li and Shaqi Luo}, journal={arXiv preprint arXiv:2606.27239v1}, year={2026}}