Robotics Research Hub
MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking

MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking

Shuaijun Liu, Chenglong Zhang, Xuhao Liu, Feiyang You, Yifan Liao, Shuyang Hao, Chaozhe Zhang, Chengyu Wu, Zhen Sun, Ningxin Su · N/A · 2026

vision reinforcement-learning
PDF arXiv Code (GitHub)

Framework

N/A

License

N/A

Stars

N/A

Summary

Human videos provide rich motion targets for humanoid learning, yet visually plausible references can still produce persistent failures under physics-based execution. These failures reveal where training supervision should change.

Abstract Summary

Human videos provide rich motion targets for humanoid learning, yet visually plausible references can still produce persistent failures under physics-based execution. These failures reveal where training supervision should change. We present MimicX, a policy-in-the-loop framework that uses execution feedback to refine video-driven humanoid motion tracking. Starting from reconstructed and retargeted motion, MimicX localizes difficult transitions and affected body regions, then jointly adapts tracking objectives and the reset curriculum for policy continuation. Repeated rollout verification selects execution-priority improvements subject to tracking guards. Across four core video tasks, MimicX consistently improves tracking accuracy and Robust Execution Horizon relative to the Fixed Reference baseline. Task-averaged results show a 25.7% reduction in body-tracking error and a 255.6% increase in execution horizon. Additional video, motion-reference, and collision-scene studies evaluate the method beyond the core tasks, while MimicX-HLoop accelerates feedback through heterogeneous execution. Overall, MimicX turns policy failure into actionable supervision for deciding what to refine and which refinement to retain.

Key Points

  • Human videos provide rich motion targets for humanoid learning, yet visually plausible references can still produce persistent failures under physics-based execution.
  • These failures reveal where training supervision should change
  • We present MimicX, a policy-in-the-loop framework that uses execution feedback to refine video-driven humanoid motion tracking.
  • Starting from reconstructed and retargeted motion, MimicX localizes difficult transitions and affected body regions, then jointly adapts tracking objectives and the reset curriculum for policy continuation.
  • Repeated rollout verification selects execution-priority improvements subject to tracking guards

Policy-in-the-Loop Supervision Refinement

The outer loop uses execution errors to revise the supervision of a tracking policy. Its input is a motion reference and a current policy; its output is the policy retained after repeated execution verification. Given a monocular video, GVHMR reconstructs world-grounded SMPL-X motion and GMR retargets it to the humanoid ( Shen et al., 2024 ; Pavlakos et al., 2019 ; Araujo et al., 2025 ) .

Policy-in-the-Loop Supervision Refinement — MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking
Figure 3: Verified updates with heterogeneous execution. Repeated rollouts rank candidates by completion, reliable motion length, and tracking quality. A body-error guard protects the current policy when no candidate qualifies. The same verification workload can run sequentially or through MimicX-HLoop. Its scheduler dispatches ready rollouts to GPUs and completed trajectories to CPU diagnosis workers, overlapping independent work while respecting dependencies. Reports are joined before selection; inputs, seeds, budgets, and the selection rule remain fixed. Execution order changes feedback latency while preserving the complete evidence supplied to the gate and its policy decision across all repeated evaluation seeds.

Experimental Setup

We evaluate whether execution feedback improves tracking, how the supervision components contribute, and whether verification and heterogeneous execution make refinement reliable and efficient. The core tasks are Tennis Swing, Football Juggling, Dance Sequence, and Kung Fu Sequence on a 29-DoF Unitree G1 in MuJoCo/MjLab. Simulation runs at 200 Hz and the PPO policy at 50 Hz.

Experimental Setup — MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking
(a) Tennis: recovered scene context

Results

MimicX improves body-tracking error and Robust Execution Horizon in all twelve task-matched comparisons against Fixed Reference (Figure 5 , Table 1 ). Task-level averaging yields a 25.7% reduction in body error and a 255.6% increase in horizon. Tennis reaches full-horizon success in all nine evaluations.

These results show that execution-conditioned supervision improves both motion fidelity and progress through demanding transitions. The paired results show that the gains extend across error channels (Table 11 ).

Results — MimicX: Policy-in-the-Loop Supervision Refinement for Video-Driven Humanoid Motion Tracking
(b) Forest: collision-scene execution

Sources and Demonstrations

Figures are reproduced from Liu et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

4D Radar Perception Algorithms for Autonomous Driving: A Review
arXiv preprint

4D Radar Perception Algorithms for Autonomous Driving: A Review

Xumin Wu, Jun Zhou, Jilin Mei et al.

Research on 4D millimeter-wave radar perception algorithms has flourished in recent years, extending from signal processing and object detection to semantic segmentation, motion estimation, occupancy prediction, and dynamic scene reconstruction.

vision reinforcement-learning benchmark
PDF Beginner