Robotics Research Hub
OMG: Omni-Modal Motion Generation for Generalist Humanoid Control

OMG: Omni-Modal Motion Generation for Generalist Humanoid Control

Siqiao Huang, Kun-Ying Lee, Dongming Qiao, Guanqi He, Zhenyu Wang, Yitang Li, Shaoting Zhu, Hang Zhao · N/A · 2026

humanoid diffusion-models generation
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Humanoid whole-body control has made significant progress in recent years, yet existing approaches remain limited to few-skill policies with heavy reward engineering, or motion trackers that are difficult to extend to new input modalities.

Abstract Summary

Humanoid whole-body control has made significant progress in recent years, yet existing approaches remain limited to few-skill policies with heavy reward engineering, or motion trackers that are difficult to extend to new input modalities. We argue that the key to general-purpose humanoid control is to build a scalable brain, a module capable of reasoning with diverse conditioning modalities, atop a reactive motion tracking cerebellum, mirroring the hierarchical structure of biological motor systems. Two challenges arise in realizing this vision: acquiring a vast amount of high-quality data to achieve general purpose control, and equipping the generator with the capability to condition on compositional, extensible multi-modal inputs. We present OMG, which addresses these challenges with a meticulous data curation, filtering and labeling pipeline, as well as a diffusion-based motion generation backbone that conditions on language, audio, and human reference motions. Extensive experiments validate OMG as an omni-modal whole-body controller exhibiting state-of-the-art performance, model scaling behavior and efficient adaptation to new distributions and modalities, marking a concrete step toward foundation models for humanoid robots.

Key Points

  • Introduces OMG, a diffusion-based omni-modal motion generator for generalist humanoid control.
  • Conditions on language, audio, and human reference motions through a scalable hierarchical architecture.
  • Employs meticulous data curation and labeling pipeline to support general-purpose control.
  • Exhibits state-of-the-art performance, model scaling, and efficient adaptation to new modalities.

OMG-Data : Scaling Multi-Modal Data for Humanoid Motion Generation

A central obstacle to scaling humanoid whole-body motion generation is the lack of a unified, robot-executable, and multi-modal motion corpus. Existing motion datasets are highly fragmented, and datasets differ substantially in their available label modality, motion quality, and annotation granularity.

OMG-Data : Scaling Multi-Modal Data for Humanoid Motion Generation — OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
Figure 2: Dataset Statistics of OMG-Data . We curate a large-scale omni-modal humanoid motion corpus by aggregating heterogeneous datasets and unifying them into the Unitree G1 motion space. Left: processed data statistics across conditioning modalities and source datasets. Right: representative conditioning modalities, including language, audio, and human reference motions.

OMG-DiT : Unified Diffusion Backbone for Motion Generation

Equipped with this large-scale multi-modal motion corpus, we instantiate OMG-DiT , a unified diffusion backbone that maps heterogeneous control modalities into robot-executable whole-body trajectories in real time.

The key design principle is to decouple the motion prior from the condition interface : a shared denoising backbone maps the distribution of feasible motions, while modality-specific encoders translate high-level intents into steerable conditions over the shared manifold.

OMG-DiT : Unified Diffusion Backbone for Motion Generation — OMG: Omni-Modal Motion Generation for Generalist Humanoid Control
Figure 4: Real-World Omni-Modal Control. OMG generates diverse Unitree G1 motions across various conditioning modalities in real time, executable in the real world.

Experiments

Our experiments aim to answer the following questions: (1) How does OMG compare against previous hierarchical humanoid control frameworks, such as graphics-based generation plus retargeting; (2) Does pretraining lead to more efficient adaptation to new modalities/datasets compared to training from scratch; and (3) Does the shared diffusion backbone exhibit foundation-model-like properties, including data scaling and zero-shot composition of control signals?

We evaluate OMG in two regimes: pretrained omni-modal motion generation and downstream f...

Sources and Demonstrations

Figures are reproduced from Huang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation
arXiv preprint

Accelerating and Scaling MPC-Guided Reinforcement Learning for Humanoid Locomotion and Manipulation

Junheng Li, Liang Wu, Sergio A. Esteban et al.

In humanoid motion control, model predictive control (MPC) offers physically grounded prediction and constraint handling, while reinforcement learning (RL) enables robust whole-body skills through large-scale simulation. However, using MPC inside RL often requires time-consuming problem construction or excessive training overhead, making such frameworks difficult to justify in practice. This work studies efficient training-time MPC guidance for humanoid locomotion and manipulation, termed MPC-RL. We introduce a centroidal-dynamics MPC reward formulation that leverages guidance from MPC trajectories in training time. To make this practical in massively parallel RL, we develop π^nMPC, a parallel-in-horizon and construction-free batched GPU MPC solver that operates directly on time-varying dynamics to avoid high memory usage and pre-compilation. Through a variety of comparative studies and hardware validations, we have found that MPC-RL achieves superior performance in locomotion and manipulation skills.

humanoid reinforcement-learning model-predictive-control locomotion manipulation
Code PDF Advanced
GitHub ★ —