LadderMan: Learning Humanoid Perceptive Ladder Climbing

Siheng Zhao, Yuanhang Zhang, Ziqi Lu, Pieter Abbeel, Rocky Duan, Koushil Sreenath, Yue Wang, C. Karen Liu, Guanya Shi · N/A · 2026

Summary

Abstract Summary

Humanoid robots hold great promise for operating in human-centered environments, yet ladder climbing remains one of the most challenging tasks due to sparse footholds and handholds, complex whole-body coordination, and sensitivity to perception and control errors. We present extbf{LadderMan}, a unified system that enables humanoid robots to robustly climb diverse ladders and perform manipulation under such constrained conditions. Our climbing policy is built on a scalable two-stage learning pipeline, where we use hybrid motion tracking to learn multiple climbing experts from a single reference motion, and distill these experts into a unified depth-based visuomotor climbing policy via hybrid imitation and reinforcement learning. To enable real-world deployment, we leverage vision foundation models to bridge the sim-to-real gap in depth perception. Building on the learned climbing policy, we further train a separate manipulation policy using a dual-agent formulation, allowing stable on-ladder manipulation via teleoperation. Experiments demonstrate that LadderMan achieves robust ladder climbing across a wide range of geometries, successfully transfers to real-world hardware in a zero-shot manner, and supports various manipulation tasks under challenging ladder constraints. Video results are available at https://ladderman-robot.github.io .

Key Points

Demonstrates humanoid robot control on real hardware
Addresses dexterous manipulation challenges
Uses simulation or synthetic data for training
Uses knowledge distillation to compress models
Leverages 3D geometric reasoning for perception
Incorporates tactile or force feedback for robust interaction
Enables complex whole-body climbing behaviors
Learns multimodal motion skills from human demonstrations

Method

Our goal is to learn a visuomotor system that enables humanoid robots to robustly climb diverse ladders and further perform manipulation under such constrained settings.

This problem is particularly challenging due to four coupled factors: (1) ladder climbing requires precise whole-body coordination with tightly coupled hand-foot contacts, (2) ladder geometries vary significantly in inclination and rung spacing, making generalization difficult, (3) perception and control errors during sim-to-real on thin and sparse structures such as ladder run...

Method — LadderMan: Learning Humanoid Perceptive Ladder Climbing — Figure 2: Overview of LadderMan. Starting from a single reference motion, we first learn multiple expert climbing policies for different ladder geometries using hybrid motion tracking. These experts are then distilled into a unified visuomotor policy through hybrid DAgger and RL. Building on the climbing policy, we further train a dual-agent manipulation policy for stable on-ladder teleoperation. To bridge the sim-to-real perception gap, we use VFM for robust real-world deployment.

Experimental Results

We evaluate LadderMan through extensive simulation and real-world experiments to answer: Q1: Can LadderMan robustly perform climbing and manipulation across diverse ladders? Q2: Does the learned dual-agent policy improve manipulation stability under ladder constraints? Q3: Can hybrid motion tracking learn expert policies across diverse ladders from single reference?

Q4: Which system components are critical for robust sim-to-real transfer? We use IsaacSim [ 16 ] for policy training and evaluation.

Experimental Results — LadderMan: Learning Humanoid Perceptive Ladder Climbing — Figure 3: We bridge the visual gap between simulation and real world by applying rung-focused masking, vision foundation model [ 27 ] , and minimalist noise augmentation.

Sources and Demonstrations

Figures are reproduced from Zhao et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.