Behavioral Cloning Mystery

Summary

Abstract Summary

Behavioral cloning (BC), despite its simplicity, exhibits many counterintuitive phenomena in the real world. For example, the performance of BC often keeps increasing as the model overfits more to the dataset, and fully closed-loop policies often completely fail without action chunking. Unfortunately, properly studying these anecdotal phenomena ("behavioral cloning mysteries") is challenging: in the real world, datasets and experiments are costly and not fully controllable; in simulation with synthetic data, these phenomena are often not easily observed partly due to the discrepancy between scripted policies and human demonstrations. In this work, we propose OCBench, a robotic manipulation benchmark with controllable scripted policies that have similar properties to human demonstrations. We show that, by mimicking key properties of human demonstrations, OCBench reproduces many anecdotal BC-related phenomena in controlled settings. With its GPU-accelerated environments and scripted policies, we demonstrate how OCBench enables scientific studies of previously reported BC-related phenomena by analyzing and refuting various hypotheses. Project page: https://seohong.me/projects/ocbench

Key Points

Behavioral cloning (BC), despite its simplicity, exhibits many counterintuitive phenomena in the real world.
For example, the performance of BC often keeps increasing as the model overfits more to the dataset, and fully closed-loop policies often completely fail without action chunking.
Unfortunately, properly studying these anecdotal phenomena ("behavioral cloning mysteries") is challenging: in the real world, datasets and experiments are costly and not fully controllable; in simulation with synthetic data, these phenomena are often not easily observed partly due to the discrepancy between scripted policies and human demonstrations.
In this work, we propose OCBench, a robotic manipulation benchmark with controllable scripted policies that have similar properties to human demonstrations.
We show that, by mimicking key properties of human demonstrations, OCBench reproduces many anecdotal BC-related phenomena in controlled settings.

Motivation

Behavioral cloning, despite its simplicity, exhibits several counterintuitive phenomena (“mysteries”), especially when using real-world data in robotics and autonomous driving. Below are some of these phenomena that have been anecdotally reported in the literature. Mystery 1: Overfitting is not necessarily bad.

OCBench: A controllable robotic manipulation benchmark

The main goal of OCBench is to provide a diverse set of tasks and datasets generated by controllable scripted policies that have similar structural properties to human demonstrations, so that researchers can scientifically study diverse BC- (and potentially RL-) related phenomena in this benchmark.

Moreover, we aim to implement both the tasks and data-generating scripts of OCBench in GPU-accelerated MuJoCo (MJWarp) to enable highly efficient data generation and simulation.

OCBench: A controllable robotic manipulation benchmark — Behavioral Cloning Mystery — Figure 2: OCBench provides $28$ tasks across five types of environments.

Studying behavioral cloning in OCBench

Now, we study the four mysteries mentioned in Section 3 , based on our reproductions in OCBench. Here, our goal is not to completely demystify these phenomena, which we leave for future work on OCBench. Rather, we aim to demonstrate examples of how OCBench’s controlled, GPU-accelerated scripts enable scientific studies of them. Behavioral cloning methodology.

Closing remarks

In this work, we have introduced OCBench, a controllable robotic manipulation benchmark, accompanied by GPU-accelerated scripted policies that have similar structural properties to human demonstrations. Throughout our analyses, we have experimentally shown that OCBench reproduces diverse real-world BC-related phenomena and enables their scientific studies via its controllable and efficient data generation scripts.

Sources and Demonstrations

Figures are reproduced from Park et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.