CUBic: Coordinated Unified Bimanual Perception and Control Framework

Summary

Recent advances in visuomotor policy learning have enabled robots to perform control directly from visual inputs.

Abstract Summary

Recent advances in visuomotor policy learning have enabled robots to perform control directly from visual inputs. Yet, extending such end-to-end learning from single-arm to bimanual manipulation remains challenging due to the need for both independent perception and coordinated interaction between arms. Existing methods typically favor one side -- either decoupling the two arms to avoid interference or enforcing strong cross-arm coupling for coordination -- thus lacking a unified treatment. We propose CUBic, a Coordinated and Unified framework for Bimanual perception and control that reformulates bimanual coordination as a unified perceptual modeling problem. CUBic learns a shared tokenized representation bridging perception and control, where independence and coordination emerge intrinsically from structure rather than from hand-crafted coupling. Our approach integrates three components: unidirectional perception aggregation, bidirectional perception coordination through two codebooks with shared mapping, and a unified perception-to-control diffusion policy. Extensive experiments on the RoboTwin benchmark show that CUBic consistently surpasses standard baselines, achieving marked improvements in coordination accuracy and task success rates over state-of-the-art visuomotor baselines.

Key Points

Proposes Coordinated Unified Bimanual Perception and Control Framework
Introduces a new dataset or benchmark
Provides open-source code or data

Preliminaries

Problem formulation will consider a task $\mathcal{T}$, where there are $N$ expert demonstrations $\{\tau_{i}\}_{i=1}^{N}$. Each demonstration $\tau_{i}$ is a sequence of state-action pairs. We formulate robot imitation learning as an action sequence prediction problem, training a model to minimize the error in future actions conditioned on historical states.

Method

Our proposed CUBic framework aims to achieve intrinsically coordinated bimanual manipulation by learning a unified perceptual representation that bridges perception and control.

The framework integrates unidirectional perception aggregation to fuse multi-view sensory observations into arm-specific perceptual tokens, bidirectional perception coordination to establish mutual perceptual awareness within a shared codebook space while preserving arm-level autonomy, and a unified perception-to-control diffusion policy that transforms the coordinated...

Method — CUBic: Coordinated Unified Bimanual Perception and Control Framework — Figure 3 : Training Recipe. We employ a two-stage training paradigm to progressively cultivate collaborative perception and control. In the first stage, actions for each arm are decoded independently, enabling the model to acquire collaborative strategies at the perception level. In the second stage, the decoding processes for both arms are fused while the perception module is frozen, thereby preserving the pre-trained perceptual collaboration capabilities and facilitating the learning of action collaboration at the control level. This progressive approach ultimately yields a unified framework for perception and action collaboration.

Experiment

In Sections 5.1 and 5.2 , we evaluate the manipulation capability of our proposed CUBic by presenting the experimental settings and results from both simulation [ 37 ] and real-world tasks, respectively. The effectiveness of each component is validated through an ablation study in Section 5.3 . Benchmark.

We use the RoboTwin [ 37 ] benchmark suite, which is widely adopted for evaluating bimanual tasks [ 54 , 1 , 33 ] . RoboTwin benchmark is built on ManiSkill [ 20 ] and features more complex scenarios.

Experiment — CUBic: Coordinated Unified Bimanual Perception and Control Framework — Figure 4 : Visualization in RoboTwin. As illustrated, compared with Diffusion Policy (DP), CUBic demonstrates superior performance in object localization and precise single-arm grasping, while maintaining strong coordination in bimanual manipulation scenarios.

Sources and Demonstrations

Figures are reproduced from Wang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.