ACT: Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware

Action Chunking with Transformers (ACT) addresses a classic problem in robot imitation learning: how to get a robot to perform precise, high-frequency, bimanual tasks such as threading a zip-tie or closing a chip bag. The key insight is that predicting a *sequence* of actions at once (a "chunk") and then replaying that chunk open-loop is far more effective than predicting one action at a time, because it reduces the compounding error of a closed-loop single-step policy while still allowing reactive behavior. ACT uses a standard CVAE (Conditional Variational Autoencoder) structure: a transformer encoder processes multi-view RGB images into keys and values, and a transformer decoder generates a fixed-length sequence of 14-DOF actions at 50 Hz. To further improve stability, the authors introduce *temporal ensembling*: when new observations arrive, the policy predicts a new chunk and averages overlapping actions with previously predicted chunks. This smooths out jitters without sacrificing responsiveness. Hardware-wise, ACT is paired with ALOHA (A Low-cost Open-source HArdware system), a dual-arm teleoperation rig built from two WidowX arms and a custom gripper system. The entire setup costs roughly $5,000, making it accessible to academic labs and hobbyists. The authors demonstrate the system on a wide range of tasks including threading a zip-tie, placing a teabag in a cup, and transferring objects between two hands. ACT demonstrates that with a modest amount of demonstration data (20–80 episodes), a carefully designed policy architecture can outperform more complex methods. The codebase is simple, clean PyTorch, and the ALOHA hardware bill of materials is fully documented open-source.