At a glance

  • The pipeline renders paired observations at matched end-effector poses and aligns source features to target features. Source
  • The target encoder is frozen while the source encoder is trained with an L2 feature-alignment loss. Source
  • The latent policy predicts Cartesian end-effector action chunks from observation history. Source

Evaluation scope The study isolates robot-to-robot imitation with a shared Cartesian end-effector action interface. Source

Pipeline showing paired source-target rendering, encoder alignment, and latent-policy deployment. Original figure

How it works

  1. Paired images are rendered for both robots from the same camera and randomized scene at matched TCP poses. Source
  2. DINOv2-base encoders are used; the target encoder is frozen and source features are optimized to match it. Source
  3. A conditional flow-matching policy is trained on encoded source demonstrations and deployed using target-encoder features. Source
  4. Simulation uses three manipulation tasks in RoboVerse with an Isaac Sim backend. Source
  5. Each task has 300 Franka demonstrations; each method is evaluated with 100 closed-loop rollouts per task and robot. Source
Paired simulated observations for Franka, xArm, and Sawyer under randomized scenes. Original figure

Evidence & results

  • In zero-shot simulation, UniLatent reaches 93% on xArm and 81% on Sawyer for Close Box Lid, 65% and 51% for Stack Cup into Bowl, and 89% and 77% for Kick Football. Evidence
  • In the real-world xArm-to-Franka setting, UniLatent obtains 27/30 successful target rollouts with ten target demonstrations per task, versus 23/30 for RoVi-Aug pretraining and 11/30 for target-only training. Evidence
  • With five held-out simulated backgrounds, mean success is 85.6% on xArm and 80.0% on Sawyer, compared with 93% and 81% under matched backgrounds. Evidence
Representative real-world task frames for the cross-embodiment transfer evaluation. Original figure

Limitations & boundaries

  • The evaluation targets 7-DoF single-arm robots with parallel-jaw grippers and a shared end-effector interface; substantially different morphologies or tool-specific contact require additional modeling. Source
  • The method does not guarantee robustness to unseen visual conditions; source-side visual diversity affects generalization. Source

Takeaway

Paired source-target renderings align a trainable source encoder to a frozen target encoder, allowing a policy learned from source demonstrations to consume target observations. Source

Resources

Figures: the paper's authors. Editorial source and image review: 2026-11-12. Findings are author-reported, not independently reproduced by Robotics Research Hub.

Original abstract

Robot datasets are costly to collect and are usually tied to a specific embodiment, making it difficult to reuse demonstrations across robots with different geometry, appearance, and kinematics. We introduce UniLatent, a cross-embodiment transfer framework that reduces the embodiment-induced observation gap in a shared latent space. UniLatent renders paired source-target robot observations in simulation and aligns a source visual encoder to a frozen target encoder. A policy trained on encoded source demonstrations can then run directly on target observations, with no pixel-level image translation. Across three simulation tasks and two target robot arms, UniLatent reaches up to 93% zero-shot success, exceeding the strongest pixel-translation baseline by over 30 points on average. In real-world xArm-to-Franka transfer, fine-tuning with only 10 target demonstrations per task yields 27/30 successful rollouts. Code, videos and data are available at https://selina2023.github.io/unilatent/.

Citation
@article{Ding2026_261110061,
  title     = {UniLatent: Cross-Embodiment Transfer via Latent Observation Alignment},
  author    = {Yufei Ding and Yi Du and Weiduo Yuan and Hao Jiang and Dylan Goetting and Lawrence Yunliang Chen and Daniel Seita and Yue Wang},
  year      = {2026},
  eprint    = {2611.10061},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2611.10061}
}
Share