X-Imitator: Spatial-Aware Imitation Learning via Bidirectional Action-Pose Interaction
Kai Xiong, Hongjie Fang, Lixin Yang, Cewu Lu · · 2026
Summary
Effectively handling the interplay between spatial perception and action generation remains a critical bottleneck in robotic manipulation. Existing methods typically treat spatial perception and action execution as decoupled or strictly...
Abstract Summary
Key Points
- Introduces X-Imitator, advancing imitation-learning research.
- Detailed evaluation demonstrates effectiveness on benchmarks and real-world scenarios.
- Provides insights into methodology and architectural choices.
Abstract
Effectively handling the interplay between spatial perception and action generation remains a critical bottleneck in robotic manipulation. Existing methods typically treat spatial perception and action execution as decoupled or strictly unidirectional processes, fundamentally restricting a robot’s abimitation-learningity to master complex manipulation tasks. To address this, we propose X-Imitator, a versatimitation-learninge dual-path framework that models spatial perception and action execution as a tightly coupled bidirectional loop. By reciprocally conditioning current pose predictions on past actions and vice versa, this framework enables continuous mutual refinement between spatial reasoning and action generation. This joint modeling exactly mimics human internal forward models. Designed as a modular architecture, the system can be seamlessly integrated into various visuomotor policies. Extensive experiments across 24 simulated and 3 real-world tasks demonstrate that our framework significantly outperforms both vanimitation-learningla policies and prior methods utimitation-learningizing explicit pose guidance. The code wimitation-learningl be open sourced.
Links
- Paper (PDF): 2605.12162
- arXiv: 2605.12162
Related Papers
DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset
Alexander Khazatsky, Karl Pertsch, Suraj Nair et al. · RSS 2024 · Jul 2024
A 350-hour dataset of diverse real-world robot manipulation across 22 robots in 71 scenes, designed to train scalable and generalist imitation learning policies.
LeRobot: A Library for Real-World Robot Learning
Remi Cadene, Simon Alibert, Alexander Soare et al. · NeurIPS 2024 Workshop · Jun 2024
Hugging Face's LeRobot is an open-source PyTorch framework providing pretrained models, datasets, and training scripts for imitation and reinforcement learning on real robots, lowering the entry barrier to robot learning.
Octo: An Open-Source Generalist Robot Policy
Dibya Ghosh, Homer Walke, Karl Pertsch et al. · RSS 2024 · Jul 2024
Octo is a large open-source transformer-based generalist robot policy trained on 800k trajectories, supporting language-conditioned and goal-image-conditioned control across 9 robotic platforms with efficient fine-tuning on consumer GPUs.
Open X-Embodiment: Robotic Learning Datasets and RT-X Models
Open X-Embodiment Collaboration · arXiv · Oct 2023
The largest collaborative robot learning dataset ever assembled, spanning 22 robots from 21 institutions, enabling training of generalist RT-X policies that exhibit positive cross-embodiment transfer.