Learning Safe Humanoid Navigation from Reduced Order Models

Summary

Research in humanoid robotics has achieved rapid progress in locomotion, and recent results have pushed the boundary on autonomous navigation.

Abstract Summary

Research in humanoid robotics has achieved rapid progress in locomotion, and recent results have pushed the boundary on autonomous navigation. We demonstrate that a standard single-stage RL navigation pipeline struggles to scale to multi-level and multi-story terrain, limited by the difficulty of complex humanoid terrain interactions such as stairs. To overcome this challenge, we decompose the navigation problem into two pieces. First, we train a policy operating on the reduced order dynamics but with full 3D LiDAR observations to navigate complex, multi-story terrain. We then utilize this navigation knowledge to kickstart a policy operating on the full-order humanoid dynamics, with a frozen locomotion policy in the loop. Additionally, we demonstrate that applying a Poisson safety filter to the navigation policy output recovers safety in the presence of out-of-distribution obstacles, without dropping navigation success rate. We demonstrate the resulting RoM-Nav policy on a Unitree G1, accomplishing mapless multi-floor navigation covering trials with over 10m of vertical displacement and over 100m of path length. Project page with videos https://wdc3iii.github.io/rom-nav/ .

Key Points

Research in humanoid robotics has achieved rapid progress in locomotion, and recent results have pushed the boundary on autonomous navigation.
We demonstrate that a standard single-stage RL navigation pipeline struggles to scale to multi-level and multi-story terrain, limited by the difficulty of complex humanoid terrain interactions such as stairs.
To overcome this challenge, we decompose the navigation problem into two pieces
First, we train a policy operating on the reduced order dynamics but with full 3D LiDAR observations to navigate complex, multi-story terrain.
We then utilize this navigation knowledge to kickstart a policy operating on the full-order humanoid dynamics, with a frozen locomotion policy in the loop.

Methods

The navigation policy is trained for deployment on a Unitree G1 equipped with a Mid-360 LiDAR and a downward-facing ZED Mini depth camera. The frozen locomotion policy for the robot is trained using LIP-CLF RL [ 3 ] . The LiDAR range image and depth image are both high-dimensional inputs which require significant visual processing to extract useful information.

Rather than extract these features during the RL training, we pretrain a CNN encoder for both the range image and depth image using a standard VAE approach [ 40 ] .

Methods — Learning Safe Humanoid Navigation from Reduced Order Models — Fig. 3: Environment tiles. (a) multi-story building and (b) its occupancy grid. (c) multi-level outdoor environment and (d) its occupancy grid.

Results

In this section we examine the RoM-Nav method, showing its improvement over the single-stage training approach, and justify the pretrained encoders and spawn/goal sampling. We demonstrate the effectiveness of the Poisson safety filter for safety around OOD obstacles on hardware. Finally, we deploy the method in real-world multi-floor environments. First, we evaluate the RoM-Nav method.

We compare a few methods: Single-Stage, trained directly on the humanoid via PPO for 4000 iterations.

Results — Learning Safe Humanoid Navigation from Reduced Order Models — Fig. 5: Investigation on performance with OOD obstacles. Four of ten trials are plotted (collisions marked with $\times$). (Left) Environment with in-distribution obstacles. (Middle) Environment with OOD obstacles. (Right) Environment with obstacles chosen adversarially, hanging with small LiDAR cross-section.

Sources and Demonstrations

Figures are reproduced from Compton et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.