One Demonstration Is Enough for Real-World Robotic Reinforcement Learning

Yuwan Liu, Hongze Yu, Song Liu, Yuhan Wang, Junge Zhang, Yaodong Yang, Yuanpei Chen, Ceyao Zhang · · 2026

Summary

Learning effective robot control policies on physical hardware is challenging due to costly data collection and the difficulty of reward specification.

Abstract Summary

Learning effective robot control policies on physical hardware is challenging due to costly data collection and the difficulty of reward specification. Prior work has incorporated demonstrations into reinforcement learning (RL), yet existing approaches either require large numbers of demonstrations or depend on continuous human intervention during training. To address these limitations, we present AutoSERL, a framework that leverages a single demonstration to fully automate the intervention process in real-world robot RL. The framework includes three complementary mechanisms to accomplish certain tasks: a sliding window intervention mechanism that continuously guides exploration to prevent local optima and unsafe deviations, a safety recovery mechanism that detects and corrects failure states via predefined trajectory recovery points, and an intervention termination criterion that automatically disables guidance once the policy can independently complete the task, preserving its exploration advantage. We evaluate AutoSERL on six contact-intensive manipulation tasks across two robot platforms, spanning insertion, hanging, and hinge-based tasks. AutoSERL consistently outperforms SERL initialized with 20 demonstrations, behavior cloning, and MILES -- a dedicated one-shot imitation learning baseline -- across all tasks while matching HIL-SERL, achieves 100% success rate on insertion tasks, and demonstrates improved robustness to positional variations, all from a single demonstration. Code and videos are available on our project website: https://autoserl.github.io/.

Key Points

Proposes One Demonstration Is Enough for Real-World Robotic Reinforce for robotics.
Evaluated on real or simulated robotic tasks.
Primary contribution in ar-vr.

Method

Reinforcement learning (RL) tasks can be modeled as a Markov Decision Process $\mathcal{M}=\left(\mathcal{S},\mathcal{A},\rho,P,r,\gamma\right)$, where $\mathcal{S}$ is the state space, $\mathcal{A}$ is the action space, $\rho(s_{0})$ is the initial state distribution, $P$ is the state transition probability, $r$ is the reward function, and $\gamma$ is the discount factor.

The learning objective is to maximize the expected cumulative reward. Our method is built upon SERL [ 14 ] .

Method — One Demonstration Is Enough for Real-World Robotic Reinforcement Learning — Figure 2 : Experimental setup. Left: the setup for the hanging and hinge-based tasks, consisting of a UR5 robot, an Inspire dexterous hand and two Intel RealSense D435 cameras. Right: the setup for the insertion tasks, consisting of a Franka robot and two wrist-mounted Intel RealSense D405 cameras.

Experiments

We conduct real-world experiments to evaluate the effectiveness of AutoSERL, aiming to address the following questions: (1) Does AutoSERL achieve higher training efficiency than existing baselines? (2) How robust is AutoSERL across different seeds and under positional variations? (3) How do different heuristic hyperparameter settings affect the performance of AutoSERL?

(4) What are the respective contributions of the individual components of AutoSERL to its overall performance? (5) Does AutoSERL merely imitate the demonstration trajectory, or c...

Experiments — One Demonstration Is Enough for Real-World Robotic Reinforcement Learning — Figure 3 : Overview of the experimental tasks: (A)Plug Insertion. (B)USB Insertion. (C)Hanger Suspension. (D)Correction Tape Suspension. (E)Spoon Suspension. (F)Drawer Opening.

Sources and Demonstrations

Figures are reproduced from Liu et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.