NEEDLEWORK: Offline Rewriting of Robot Data with Verified Local Stitches

Summary

Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful.

Abstract Summary

Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful. Trajectory stitching offers a way to compose these behaviors into improved training data, but identifying useful connections and verifying their feasibility is difficult in high-dimensional robot data, where many prior methods rely on low-dimensional state representations. We introduce NEEDLE, an offline dataset-augmentation algorithm that addresses these challenges by adding short, verified action bridges between recorded observations in high-dimensional robot demonstrations. First, NEEDLE identifies and creates connections that bypass suboptimal detours, broaden action coverage, and augment the original dataset with failed trajectories, using only RGB images, proprioception, and episode-level outcomes, without new environment interaction or privileged object state. Next, we present a sampling technique that incorporates accepted bridges into policy training without synthesizing intermediate images or discarding the original demonstrations, allowing policies to learn alternative actions while retaining the original dataset's coverage. On real-robot tasks, NEEDLE improves success rate over the strongest baseline on each task by an average of 21 percentage points. Videos and supplementary materials are on https://needle-work.github.io/.

Key Points

Robot demonstrations may contain useful behavior even when individual episodes are inefficient or unsuccessful.
Trajectory stitching offers a way to compose these behaviors into improved training data, but identifying useful connections and verifying their feasibility is difficult in high-dimensional robot data, where many prior methods rely on low-dimensional state representations.
We introduce NEEDLE, an offline dataset-augmentation algorithm that addresses these challenges by adding short, verified action bridges between recorded observations in high-dimensional robot demonstrations.
First, NEEDLE identifies and creates connections that bypass suboptimal detours, broaden action coverage, and augment the original dataset with failed trajectories, using only RGB images, proprioception, and episode-level outcomes, without new environment interaction or privileged object state.
Next, we present a sampling technique that incorporates accepted bridges into policy training without synthesizing intermediate images or discarding the original demonstrations, allowing policies to learn alternative actions while retaining the original dataset's coverage.

How Needle work Works

Needle answers two separate questions when rewriting a dataset: whether a stitch is useful , and whether a stitch is feasible . Candidate selection addresses the first question. An Inverse Dynamics Model (IDM) and an action-conditioned verifier address the second.

We formalize the notation in § 3.1 , the selection of useful connections in § 3.2 , action generation and verification in § 3.3 , and policy training on the rewritten dataset in § 3.4 . Additional details in Appendix D .

How Needle work Works — NEEDLEWORK: Offline Rewriting of Robot Data with Verified Local Stitches — Figure 2: From candidate connection to verified bridge. Given a source-target pair $(s,g)$, the inverse dynamics model (IDM) samples $K$ action proposals. The action-conditioned verifier predicts whether and when each proposal reaches $g$. We select the shortest accepted prefix as a bridge.

Experiments

We evaluate whether rewriting an offline robot dataset with Needle improves downstream policy performance, and, more importantly, how the rewritten data changes the behavior learned by the policy. After describing the experimental setup (§ 4.1 ), we first measure task success on real-robot and simulated manipulation tasks (§ 4.2 ).

Sources and Demonstrations

Figures are reproduced from Ren et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.