Robotics Research Hub
Continual Humanoid Motion Learning

Continual Humanoid Motion Learning

Zhewen He, Hao Huang, Geeta Chandra Raju Bethala, Chong Yu, Tao Chen, Anthony Tzes, Yi Fang · N/A · 2026

reinforcement-learning control
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered.

Abstract Summary

Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered. We study continual learning for humanoid whole-body motion, where a single controller must acquire skills from a sequential task stream without revisiting past data. We introduce Similarity-guided LoRA-PNN, a progressive neural network (PNN) policy that prevents catastrophic forgetting by construction while reusing knowledge across skills through lightweight low-rank adaptation. A two-level motion-similarity measure, built from dynamic time warping aggregated by optimal transport, decides which prior skill to build on and how much new capacity to allocate, yielding strong forward transfer and large efficiency gains. Across six sequentially learned skill categories, our similarity-guided LoRA policy attains the best forward transfer (0.125 vs. 0.079) and the highest average accuracy among all methods, while saving up to 94.5% of trainable parameters and 40.8% of training time. The resulting controller reaches 96.13% sim-to-sim transfer and is deployed on a physical Unitree G1. Our code is available at https://anonymous.4open.science/r/continual-humanoid-learning-35D3.

Key Points

  • Humanoid whole-body controllers can now track a diverse set of dynamic motions, but they are typically trained offline and then frozen, so teaching such a controller a new skill tends to erode the skills it already mastered.
  • We study continual learning for humanoid whole-body motion, where a single controller must acquire skills from a sequential task stream without revisiting past data.
  • We introduce Similarity-guided LoRA-PNN, a progressive neural network (PNN) policy that prevents catastrophic forgetting by construction while reusing knowledge across skills through lightweight low-rank adaptation.
  • A two-level motion-similarity measure, built from dynamic time warping aggregated by optimal transport, decides which prior skill to build on and how much new capacity to allocate, yielding strong forward transfer and large efficiency gains.
  • Across six sequentially learned skill categories, our similarity-guided LoRA policy attains the best forward transfer (0.125 vs.

Method

We study continual learning for humanoid whole-body motion, where a single controller must incrementally acquire a growing set of whole-body skills ( e.g . , walking, jumping, and dancing) from a sequential curriculum in which skills are introduced one after another.

Let $\pi_{\theta}$ map humanoid observations (proprioception, root state, contact cues, and optional task commands) to actions, and let $\{\mathcal{D}_{1},\ldots,\mathcal{D}_{K}\}$ be the ordered task stream, where $\mathcal{D}_{k}$ holds the demonstrations for task $k$.

Method — Continual Humanoid Motion Learning
Figure 2: Overview of the continual whole-body locomotion learning pipeline. A collection of retargeted motions Harvey et al. (2020) is organized into a sequential stream of tasks ( e.g . , jumping, dancing, and fighting) and learned using a teacher–student framework. The teacher policy, with access to privileged information, is expanded for each new task by adding a trainable column (solid blue) via similarity-guided LoRA, while previously learned columns (dashed orange) are kept frozen to preserve earlier skills. The teacher’s actions supervise the student policy through a distillation loss, and the student is deployed in the real world

Experiments

We study continual humanoid whole-body motion learning under a task-incremental protocol with a provided task identifier. Let $\mathcal{K}=\{1,\dots,K\}$ be the task set, trained in order $1\!\rightarrow\!2\!\rightarrow\!\cdots\!\rightarrow\!K$.

After learning task $k$, we evaluate the policy $\pi_{k}$ on all tasks $i\leq k$, forming a stage-by-task matrix $A\in\mathbb{R}^{K\times K}$ with $A_{k,i}=\mathrm{Perf}(\pi_{k},i)$. Our suite contains $K=6$ tasks curated from LAFAN1 Harvey et al. (2020) and the Kungfu motion datasets Xie et al.

Experiments — Continual Humanoid Motion Learning
Figure 3: Motion-set distance matrix over the six categories (lighter = more similar)

Motion Data Format and Implementation Notes

This appendix complements Sec. 3.4 with the data format and the implementation constants of the motion-set distance. Each retargeted motion carries the following attributes: Root translation : root_trans_offset $\in\mathbb{R}^{T\times 3}$, the root position over time in a fixed world frame. Root orientation : root_rot $\in\mathbb{R}^{T\times 4}$, the quaternion root rotation over time in a fixed world frame.

Pose parameters : pose_aa $\in\mathbb{R}^{T\times J\times 3}$, axis-angle rotations for $J$ joints.

Motion Data Format and Implementation Notes — Continual Humanoid Motion Learning
Figure 4: Radar plots of CL metrics. (Left) Success rate. (Right) Negative MPJPE score. Polygons show min–max normalized values with square-root transformation for visual separation; numbers denote original metric values

Hyperparameters and Implementation Details

This appendix expands the summary in Table 3 . Unless otherwise specified, a single hyperparameter setting is shared across all tasks. The motion-distance settings are given in Appendix 6 and Table 3 . All policies are trained with PPO Schulman et al.

(2017) under a shared configuration: fixed rollout horizon, mini-batch count, and learning rate, with multiple PPO epochs per update, generalized advantage estimation, and the standard clipped policy and value objectives (Table 12 ).

Hyperparameters and Implementation Details — Continual Humanoid Motion Learning
Figure 5: Similarity-guided LoRA rank allocation. Rank assignment under the linear rule, which smoothly increases rank as task similarity decreases, and the binned rule, which maps similarity into discrete rank levels

Statements and Declarations

The authors have no competing interests to declare that are relevant to the content of this article. Zhewen He designed the method, implemented the code, conducted the simulation experiments, took part in the real-robot deployment, and wrote the first draft of the manuscript. Hao Huang advised on the design of the method and carried out the real-robot deployment together with Zhewen He and Geeta Chandra Raju Bethala.

Chong Yu, Tao Chen, Anthony Tzes, and Yi Fang provided guidance on the research and secured the funding.

Statements and Declarations — Continual Humanoid Motion Learning
Figure 6: Qualitative sim-to-sim comparison of the single distilled student on dance , fall_and_up , fight , and run : the same policy executed in Isaac Gym (left, with reference key-body markers) and MuJoCo (right)

Sources and Demonstrations

Figures are reproduced from He et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers