At a glance
- The pipeline renders paired observations at matched end-effector poses and aligns source features to target features. Source
- The target encoder is frozen while the source encoder is trained with an L2 feature-alignment loss. Source
- The latent policy predicts Cartesian end-effector action chunks from observation history. Source
Evaluation scope The study isolates robot-to-robot imitation with a shared Cartesian end-effector action interface. Source
How it works
- Paired images are rendered for both robots from the same camera and randomized scene at matched TCP poses. Source
- DINOv2-base encoders are used; the target encoder is frozen and source features are optimized to match it. Source
- A conditional flow-matching policy is trained on encoded source demonstrations and deployed using target-encoder features. Source
- Simulation uses three manipulation tasks in RoboVerse with an Isaac Sim backend. Source
- Each task has 300 Franka demonstrations; each method is evaluated with 100 closed-loop rollouts per task and robot. Source
Evidence & results
- In zero-shot simulation, UniLatent reaches 93% on xArm and 81% on Sawyer for Close Box Lid, 65% and 51% for Stack Cup into Bowl, and 89% and 77% for Kick Football. Evidence
- In the real-world xArm-to-Franka setting, UniLatent obtains 27/30 successful target rollouts with ten target demonstrations per task, versus 23/30 for RoVi-Aug pretraining and 11/30 for target-only training. Evidence
- With five held-out simulated backgrounds, mean success is 85.6% on xArm and 80.0% on Sawyer, compared with 93% and 81% under matched backgrounds. Evidence
Limitations & boundaries
- The evaluation targets 7-DoF single-arm robots with parallel-jaw grippers and a shared end-effector interface; substantially different morphologies or tool-specific contact require additional modeling. Source
- The method does not guarantee robustness to unseen visual conditions; source-side visual diversity affects generalization. Source
Takeaway
Paired source-target renderings align a trainable source encoder to a frozen target encoder, allowing a policy learned from source demonstrations to consume target observations. Source
Resources
Figures: the paper's authors. Editorial source and image review: 2026-11-12. Findings are author-reported, not independently reproduced by Robotics Research Hub.
Original abstract
Robot datasets are costly to collect and are usually tied to a specific embodiment, making it difficult to reuse demonstrations across robots with different geometry, appearance, and kinematics. We introduce UniLatent, a cross-embodiment transfer framework that reduces the embodiment-induced observation gap in a shared latent space. UniLatent renders paired source-target robot observations in simulation and aligns a source visual encoder to a frozen target encoder. A policy trained on encoded source demonstrations can then run directly on target observations, with no pixel-level image translation. Across three simulation tasks and two target robot arms, UniLatent reaches up to 93% zero-shot success, exceeding the strongest pixel-translation baseline by over 30 points on average. In real-world xArm-to-Franka transfer, fine-tuning with only 10 target demonstrations per task yields 27/30 successful rollouts. Code, videos and data are available at https://selina2023.github.io/unilatent/.
Citation
@article{Ding2026_261110061,
title = {UniLatent: Cross-Embodiment Transfer via Latent Observation Alignment},
author = {Yufei Ding and Yi Du and Weiduo Yuan and Hao Jiang and Dylan Goetting and Lawrence Yunliang Chen and Daniel Seita and Yue Wang},
year = {2026},
eprint = {2611.10061},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2611.10061}
}