HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Dairu Liu, Zekun Qi, Jiayu Zeng, Ruixi Yu, Yu Guan, Yintianrun Zhang, Xuchuan Chen, Sikai Liang, Zekai Li, Chenghuai Lin, Xinqiang Yu, Wenyao Zhang, He Wang, Li Yi · N/A · 2026

Summary

Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos.

Abstract Summary

Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos. Kinematic errors average per-frame pose differences but miss the physical artifacts that matter most, particularly unstable support and incorrect contacts such as foot skating and mistimed touch-downs. Meanwhile, widely used test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors. We introduce HumanTracker to make humanoid tracking evaluation both perceptually aligned and scalable. The HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis. We further propose HumanScore, a preference-aligned metric trained on 12K motion pairs containing 24K motions. Across representative state-of-the-art trackers, HumanScore better predicts human preferences and reveals contact and stability failures that kinematic metrics often miss.

Key Points

Humanoid motion tracking is central to teleoperation and whole-body imitation, yet evaluation often disagrees with what people perceive in videos.
Kinematic errors average per-frame pose differences but miss the physical artifacts that matter most, particularly unstable support and incorrect contacts such as foot skating and mistimed touch-downs.
Meanwhile, widely used test suites are small and lack the diversity needed to stress contact-rich, long-horizon behaviors.
We introduce HumanTracker to make humanoid tracking evaluation both perceptually aligned and scalable.
The HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from multiple professional performers, organized into four motion families with text labels for fine-grained diagnosis.

HumanTracker: Benchmark and Preference-Aligned Evaluation

The released HumanTracker benchmark contains approximately 153 hours of optical motion trajectories from 24 professional performers. The performers include dance teachers, fitness coaches, tennis coaches and full-time motion-capture actors. The performers and recording plan were chosen to cover both routine movement and motions that place substantially different demands on a humanoid controller.

We retarget each fitted human motion to the benchmark humanoid using General Motion Retargeting (GMR) [ 1 ] .

HumanTracker: Benchmark and Preference-Aligned Evaluation — HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark — Figure 2 : Preference collection interface. Annotators view two synchronized rollouts of the same reference segment. Display order is randomized and the available responses are Left is better , Right is better , Similar and Cannot compare .

Experiments

We evaluate GMT, TWIST2, SONIC and Humanoid-GPT through the standardized protocol of Sec. 3.2 . Each policy retains its native observation and action-processing stack, while all methods receive the same retargeted reference motions and are measured by the same evaluator. We apply the whole-body metric defined in Sec. 3.2 to every tracker. None of the trackers is trained or fine-tuned on HumanTracker.

Sources and Demonstrations

Figures are reproduced from Liu et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.