Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer

Humanoid-Gym is a reinforcement learning framework specifically built to close the reality gap for bipedal robots. Built atop NVIDIA Isaac Gym (Preview 4), it provides a curated set of humanoid-specific reward functions, terrain curricula, and domain-randomization schedules that collectively enable zero-shot sim-to-real transfer. The framework ships with example policies for XBot-S (1.2 m) and XBot-L (1.65 m), two commercial humanoid platforms from RobotEra. The training paradigm centers on proximal policy optimization (PPO) in massively parallel simulation (4096 environments). The unique contributions are in how the reward functions implicitly encode humanoid priors—penalizing unnatural knee hyperextension, encouraging symmetrical gait, and shaping trunk posture—such that the learned policy remains stable when transferred to hardware without any real-world fine-tuning. A notable design choice is the built-in sim2sim transfer pipeline. Researchers can first train in Isaac Gym and then test the exact same policy within MuJoCo environments, allowing coarse validation of dynamics mismatch before risking hardware. Terrain curriculum and actuator noise injection are the primary robustification tools, gradually increasing terrain roughness and randomizing motor torque bandwidth to promote robust gaits. For researchers and startups entering the humanoid space, Humanoid-Gym offers a plug-and-play training package with well-documented URDF setup, reward templates, and deployment scripts. The included domain-randomization recipes are tuned over extensive real-world experiments and represent practical best practices rather than theoretical guidelines. This makes it one of the most pragmatic toolkits for anyone seeking to deploy learned walking on their own humanoid platform.