Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Featured

Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, Shuran Song · Columbia University, University of Tokyo, Toyota Research Institute · 2023

Framework

PyTorch

License

MIT

Stars

4,194

Summary

A behavior cloning approach that models robot policies as conditional diffusion processes, enabling multimodal action distributions and smooth action sequences.

Abstract Summary

Diffusion Policy rethinks behavior cloning by treating policy learning as a generative modeling problem. Rather than predicting a single deterministic action or fitting a unimodal Gaussian, the method trains a denoising diffusion model to generate a coherent action sequence conditioned on the current visual observation. Starting from random noise, the model iteratively refines its output over a series of denoising steps until a smooth, feasible action chunk emerges. This formulation elegantly solves a long-standing problem in imitation learning: how to represent multimodal action distributions without suffering from averaging artifacts. In traditional behavior cloning, when the demonstration dataset contains multiple valid strategies for the same scenario—such as pushing an object from the left versus the right—a deterministic policy tends to average these options into a meaningless compromise. Diffusion Policy avoids this by learning the full distribution of demonstrated actions. Because diffusion models are inherently multimodal, they can capture diverse valid strategies and sample from them at test time. Additionally, the model predicts short action horizons (e.g., 8 or 16 future timesteps) rather than isolated instantaneous actions. This chunk-based prediction leads to significantly smoother and more temporally consistent robot behavior, reducing jerk and mechanical wear. The architecture is surprisingly flexible. The authors show that the denoising network can be implemented as a convolutional U-Net, a transformer, or even a simple MLP, depending on the observation and action space. Training follows the standard conditional diffusion objective: given an observation, the model learns to reverse the noising process applied to the corresponding action chunk from the demonstration dataset. At inference time, the policy runs a fixed number of denoising iterations (typically 10 to 100) to generate the final action sequence. The paper also demonstrates that inference acceleration techniques such as DDIM can cut computational cost with minimal impact on task success. Empirical evaluation spans both simulated benchmarks and real hardware. On the classic Push-T contact-rich manipulation task, Diffusion Policy outperforms prior state-of-the-art methods including ACT and VINN by a wide margin. It also achieves strong results on the Franka Kitchen multi-task benchmark and a suite of real-world robot tasks such as pouring and slot insertion. The hardware experiments confirm that the smooth, multimodal actions generated by diffusion translate directly into higher success rates and more robust behavior on physical robots.

Key Points

  • Formulates behavior cloning as conditional diffusion over action sequences.
  • Handles multimodal action distributions, avoiding deterministic averaging artifacts.
  • Predicts action chunks (short horizons) for temporal smoothness and consistency.
  • Compatible with CNN, Transformer, or MLP denoising backbones.
  • Strong results on Push-T, Franka Kitchen, and real robot manipulation tasks.

Additional Notes

Training Tips

  • Use the provided data augmentation pipelines for vision-based tasks; they are critical for generalization.
  • Action horizon (prediction chunk length) is a key hyperparameter: longer horizons improve smoothness but increase inference cost.
  • DDIM sampling with 10-20 steps is usually sufficient for real-time control.
  • ACT (Zhao et al., 2023)
  • Implicit Behavioral Cloning (Florence et al., 2022)
  • RoboDiffusion (not a real paper, but many follow-ups exist)
Share

Related Papers

EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera

Christen Millerdurai, Shaoxiang Wang, Yaxu Xie et al. · SIGGRAPH 2026 · May 2026

Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must...

manipulation manipulation simulation
Code PDF Intermediate
Code ★ 0 May 2026
Aligning Flow Map Policies with Optimal Q-Guidance

Aligning Flow Map Policies with Optimal Q-Guidance

Christos Ziakas, Alessandra Russo, Avishek Joey Bose · arXiv · May 2026

Generative policies based on expressive model classes, such as diffusion-models and flow matching, are well-suited to complex control problems with highly multimodal action distributions. Their expressivity, however, comes at a significant inference cost:...

rl diffusion-models manipulation
PDF Advanced
No code repo May 2026
MonoDuo: Using One Robot Arm to Learn Bimanual Policies

MonoDuo: Using One Robot Arm to Learn Bimanual Policies

Sandeep Bajamahal, Lawrence Yunliang Chen, Toru Lin et al. · ICRA 2026 · May 2026

Bimanual coordination is essential for many real-world manipulation tasks, yet learning bimanual robot policies is limited by the scarcity of bimanual robots and datasets. Single-arm robots, however,

manipulation bimanual imitation-learning dataset
PDF Intermediate
No code repo Code updated: May 2026