Isaac Gym: High Performance GPU-Based Physics Simulation for Robot Learning

Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikhil Rudin, Arthur Allshire, Ankur Handa, Gavriel State · NVIDIA · 2021

Summary

A GPU-accelerated physics simulation framework that enables thousands of RL environments to run in parallel on a single GPU, eliminating CPU-GPU transfer bottlenecks.

Abstract Summary

Isaac Gym fundamentally changes the economics of reinforcement learning for robotics by moving the entire physics simulation pipeline onto the GPU. Prior to Isaac Gym, the standard workflow involved running physics on CPU (via MuJoCo, PyBullet, or Gazebo), transferring state observations to GPU for neural network inference, then sending actions back to CPU for the next simulation step. This CPU-GPU ping-pong created a severe bottleneck, limiting researchers to tens or at most hundreds of parallel environments. Isaac Gym solves this by performing all rigid-body physics computations on the GPU using NVIDIA's PhysX engine, allowing thousands of environments to run in parallel on a single workstation GPU with minimal overhead. The performance gains are dramatic. On tasks such as humanoid walking, ant locomotion, and Franka robot manipulation, Isaac Gym can simulate over 100,000 parallel environments at interactive frame rates on a single NVIDIA RTX 3090 or A100. This massive parallelism directly translates into faster training for model-free RL algorithms like PPO and SAC. Experiments in the paper show that what used to require a cluster of CPU servers can now be done on a single desktop in a fraction of the time. For example, training a humanoid policy from scratch that previously took days on CPU clusters can be completed in under an hour on GPU. Beyond raw speed, Isaac Gym introduces a unified tensor-based API that simplifies the development of batched RL environments. Observations, actions, rewards, and dones are all represented as PyTorch tensors living in GPU memory, making it trivial to plug in modern deep learning frameworks without any data marshalling code. The system also supports direct force and contact sensor readings, domain randomization for sim-to-real transfer, and integration with the USD scene description format for complex environment authoring. The physics solver is deterministic and reproducible, which is crucial for debugging RL algorithms and comparing research results. The release of Isaac Gym (and its successor Isaac Sim) has had an outsized impact on the robotics RL community. It enabled projects like Legged Gym, OmniDrones, and many dexterous manipulation benchmarks that would have been computationally infeasible under the old CPU-centric paradigm. The paper benchmarks a wide range of classic control and robotics tasks, demonstrating that GPU simulation does not sacrifice physical accuracy compared to CPU engines for the regimes relevant to RL. For any lab working on policy learning for locomotion or manipulation, understanding and using Isaac Gym is now essentially a prerequisite.

Key Points

End-to-end GPU physics simulation using PhysX, eliminating CPU-GPU transfer bottlenecks.
Scales to 100,000+ parallel environments on a single GPU.
Unified tensor API (PyTorch) for observations, actions, and rewards.
Compatible with PPO, SAC, and other standard RL algorithms.
Enabled a new generation of large-scale RL benchmarks (Legged Gym, etc.).

Massively Parallel Reinforcement Learning

Current (on-policy) reinforcement learning algorithms are divided into two parts: data collection and policy update. The policy update, which corresponds to back-propagation for neural networks, is easily performed in parallel on the GPU. Parallelizing data collection is not as straightforward. Each step consists of policy inference, simulation, reward, and observation calculation.

Task Description

A quadruped robot must learn to walk across challenging terrain, including uneven surfaces, slopes, stairs, and obstacles, while following base-heading and linear-velocity commands. We conduct most of the simulation and real-world deployment experiments on the ANYbotics ANYmal C robot.

However, in simulation, we demonstrate the broader applicability of the approach by additionally training policies for ANYmal B, ANYmal C with an attached arm, and the Unitree A1 robots. The terrains are selected to be representative of real-world environments.

Task Description — Isaac Gym: High Performance GPU-Based Physics Simulation for Robot Learning — (b)

Results

In this section, we study the effects of the number of parallel robots on the final performance of the policy. In order to use the total reward as a single representative metric, we have to remove the curriculum, otherwise a more performant policy sees its task difficulty increase and consequently a decrease in the total reward.

As such, we simplify the task by reducing the maximum step size of stairs and obstacles and directly train robots on the full range of difficulties.

Results — Isaac Gym: High Performance GPU-Based Physics Simulation for Robot Learning — (c)

Sources and Demonstrations

Figures are reproduced from Makoviychuk et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.