Legged Gym: A Framework for Massively Parallel Reinforcement Learning of Legged Locomotion

Legged Gym: A Framework for Massively Parallel Reinforcement Learning of Legged Locomotion

Featured

Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter · ETH Zurich, Robotic Systems Lab · 2022

Framework

PyTorch / Isaac Gym

License

Other (NOASSERTION)

Stars

2,971

Summary

Legged Gym is an open-source training framework built on NVIDIA Isaac Gym that enables massively parallel deep RL for legged robot locomotion, achieving policy training times of minutes rather than days.

Abstract Summary

Legged Gym emerged from a simple but powerful observation: deep reinforcement learning for legged locomotion does not need to take days on server clusters if the simulation itself is massively parallelized on a single GPU. Built as a high-level wrapper around NVIDIA Isaac Gym, Legged Gym provides a complete training stack—including environment definitions, reward functions, curriculum strategies, and policy architectures—for training quadruped and humanoid robots in simulation. By running thousands of environments simultaneously on a single GPU, the framework compresses training times from days to minutes without sacrificing policy quality. The framework ships with environments for several popular robot platforms, including Unitree A1, Go1, and ANYmal, as well as the bipedal robot Cassie. Each environment includes a carefully tuned reward function and terrain curriculum that progressively increases task difficulty as the policy improves. The terrain curriculum is particularly noteworthy: it is inspired by game design principles, procedurally generating increasingly challenging elevation maps, stairs, and obstacles to prevent premature convergence and encourage robust gaits. The authors show that policies trained with this curriculum transfer directly to real-world hardware with minimal domain randomization, a testament to the physical fidelity of the Isaac Gym simulator and the robustness of the learned gaits. From a software engineering perspective, Legged Gym is clean, modular, and highly extensible. Environment parameters are managed through structured YAML configs, making it easy to sweep reward weights, terrain properties, and robot dynamics. The training loop uses PyTorch and leverages Isaac Gym’s GPU-based physics backend, so all simulation rollouts happen on the GPU with minimal CPU-GPU transfer overhead. The framework also supports real-time visualization, logging, and checkpointing, enabling rapid iteration and debugging. Perhaps the most impactful result is the training speed benchmark. On a single NVIDIA RTX 3090, Legged Gym trains a flat-terrain walking policy for ANYmal in under four minutes, and an uneven-terrain policy in under twenty minutes. This represents orders of magnitude speedup compared to traditional CPU-based simulation pipelines, democratizing access to legged robot learning for labs without large compute budgets. The codebase has become a de facto standard in the legged locomotion community, with hundreds of derivative works extending it to new robots, new tasks, and new learning algorithms.

Key Points

  • High-level wrapper around Isaac Gym enabling thousands of parallel legged robot environments on a single GPU.
  • Built-in terrain curriculum inspired by game design for progressively harder locomotion challenges.
  • Pre-configured environments for Unitree A1/Go1, ANYmal, and Cassie with tuned reward functions.
  • Training times reduced from days to minutes: flat terrain in under 4 minutes on an RTX 3090.
  • Sim-to-real transfer validated on physical ANYmal and A1 robots without extensive domain randomization.

Additional Notes

Training Tips

  • Start with the default terrain curriculum; do not flatten terrain until the policy converges on flat ground.
  • Reward tuning is critical for emergent gait styles; small changes in velocity tracking weight yield dramatically different behaviors.
  • For sim-to-real transfer, add joint angle and motor model domain randomization in the final training stages.
  • RSL-RL (Rudin & Hoeller, open-source library)
  • Isaac Gym (Makoviychuk et al., 2021)
  • DribbleBot (Ji et al., 2023)
Share

Related Papers

RSL-RL: A Fast and Flexible Reinforcement Learning Library for Legged Robots

RSL-RL: A Fast and Flexible Reinforcement Learning Library for Legged Robots

Nikita Rudin, David Hoeller, Marco Hutter · Open Source Library · Jun 2022

RSL-RL is a lightweight, high-performance PyTorch library implementing PPO tailored for locomotion. Used as the default RL backend in Legged Gym, it enables training thousands of agents in parallel on GPU.

locomotion rl simulation quadruped
Code Intermediate
GitHub ★ 2,624 Code updated: May 2026
A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms

A Heterogeneous Architecture for Robot RL Beyond GPU-Dominant Paradigms

Yufei Jia, Zhanxiang Cao, Mingrui Yu et al. · arXiv preprint · May 2026

Simulation-based RL for contemporary robot control is increasingly organized around GPU-resident simulation: physics, rollout collection, and learning are placed on a single GPU-centric execution path

reinforcement-learning simulation system-design locomotion
PDF Intermediate
No code repo Code updated: May 2026