Robotics Research Hub
Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation

PhysicalRSI Team, Danjiao Ma, Enhui Ma, Haohan Liu, Heng Jia, Hui Shan, Jianhua Xu, Jiahuan Zhang, Jiangdi Xu, Kaiwen Guo, Kaicheng Yu, Linwei Zhang, Liyang Jin, Maochun Luo, Pengyao Niu, Shiwen Li, Shuangyu Feng, Tong Zhang, Tianheng Wang, Xin Wang, Xiangru Huang, Yongqiang Huang, Zhaozhi Wang, Zijian Ma · N/A · 2026

vision reinforcement-learning learning-from-demonstration benchmark
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution.

Abstract Summary

Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution. We present\textbf{Awomo-SimDataEngine}, an agentic system that connects asset and scenegeneration to robot demonstration synthesis. Shared asset services providerigid and articulated objects, including structure-grounded part and jointgeneration with ISArt. Scene generation supports two complementary routes:Unravel reconstructs editable scenes from images, while SimForge buildssingle-room and multi-room environments from text. A graph-native harnesscoordinates construction, validation, andbounded repair, routing failures to the responsible module while retainingunaffected scene state. PolicyForge binds validated worlds to tasks and robotembodiments to produce replayable demonstrations. Evaluations cover assetgeometry, scene quality, and downstream policy learning. On MuJoCo-basedLIBERO-Plus, co-training with Isaac Sim demonstrations improves the overallsuccess rate of a World-Action Model (WAM) from $77.17\%$ to $89.43\%$. Goal and spatialsuccess improve by $31.66$ and $6.25$ percentage points, respectively.These results support the utility of the generated data for cross-simulatorpolicy training, with more limited gains on long-horizon tasks.

Key Points

  • Generating useful robot-training data requires more than visually plausiblescenes: objects must support interaction, placements must remain physicallyvalid, and tasks must admit repeatable execution.
  • We present\textbf{Awomo-SimDataEngine}, an agentic system that connects asset and scenegeneration to robot demonstration synthesis.
  • Shared asset services providerigid and articulated objects, including structure-grounded part and jointgeneration with ISArt.
  • Scene generation supports two complementary routes:Unravel reconstructs editable scenes from images, while SimForge buildssingle-room and multi-room environments from text.
  • A graph-native harnesscoordinates construction, validation, andbounded repair, routing failures to the responsible module while retainingunaffected scene state.

Overview

Awomo-SimDataEngine connects simulation-ready asset generation, text- and image-driven scene generation, and automated demonstration synthesis through a Graph-Native Harness. Sections 4 – 5 describe ISArt, Unravel, and SimForge; Sections 6 – 7 describe their orchestration and the PolicyForge interface. Module-level asset and scene evaluations are reported separately from downstream policy co-training results.

The input is $C=\{T_{\textup{text}},I,Q\}$: a scene description, one image, and an optional task request.

Overview — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 2 : ISArt articulated-asset generation. Structural planning supplies part and kinematic priors for learned geometry generation. Agentic binding constructs a URDF, and rendered simulation feedback guides parameter repair, structural re-planning, or geometry regeneration.

Asset Generation

Asset generation converts an object request into a validated simulator asset. Given text, an image, or both, the module obtains candidates through retrieval, image-to-3D generation, specialized generation, or procedural modeling. Object-centric references remove distracting scene content and provide appearance and structural cues.

Each branch returns geometry and available semantics, orientation, scale, articulation, and physical information to the shared validation process in Section 6 .

Asset Generation — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 3 : Unravel single-view scene generation. Verified removals expose objects and collect assets and point evidence. Reverse-order recomposition registers each asset to its attributed cluster, followed by support-aware settling and a final audit.

Scene Generation

Awomo-SimDataEngine supports two parallel scene-generation routes. Unravel reconstructs an editable scene from a single image, optionally accompanied by a description; SimForge constructs single-room scenes and multi-room houses from text. Neither route requires the other as a preprocessing stage.

Both use the asset services in Section 4 , retain object and articulation identities, and produce worlds for validation and downstream demonstration synthesis.

Scene Generation — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 4 : Qualitative examples of PolicyForge robot interactions across different embodiments and scene configurations. The $4\times 4$ grid shows bedroom scenes in the top two rows and coffee-shop scenes in the bottom two rows. Examples illustrate variation in robot embodiment, object arrangement, and viewpoint.

Graph-Native Harness for Execution and Repair

The Graph-Native Harness coordinates asset generation, Unravel reconstruction, SimForge scene construction, simulator packaging, and PolicyForge demonstration synthesis. Each execution edge specifies its inputs, responsible module, validation requirements, and records needed to resume or repair the result. Figure 1 shows these system connections.

The execution graph organizes module calls and recovery for both SimForge’s text-driven construction and Unravel’s image-grounded recomposition.

Graph-Native Harness for Execution and Repair — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 5 : PolicyForge pipeline. Given a scene, task description, robot embodiment, and target episode count $N$, an LLM analyzes the task and decomposes it into atomic tasks when needed. A task manager selects suitable execution backends. Simulation execution in Isaac Sim produces synchronized observations, states, and actions, which are assembled into episodes. Episodes that pass task-success validation are exported in LeRobot format. The microwave images illustrate an ISArt input asset and its simulated manipulation.

Policy Generation via Automated Demonstration Synthesis

Given a validated Awomo-SimDataEngine world and a task contract, we propose PolicyForge , which provides a fully automated pipeline that converts a user-level task specification into large-scale, multi-embodiment demonstration data ready for policy learning.

The pipeline operates within NVIDIA Isaac Sim and is designed to be embodiment-agnostic: it supports a growing roster of robotic platforms including Franka, Piper dual-arm, Agibot G1, and Agibot G2, each interacting with diverse objects across procedurally generated scenes.

Policy Generation via Automated Demonstration Synthesis — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 6 : Best-checkpoint co-training comparison. Best observed overall WAM success on LIBERO-Plus up to and including 13K steps. SimData-Ultra gives the highest score.

Experiments and Results

We evaluate whether PolicyForge demonstrations generated in Isaac Sim improve a World-Action Model (WAM) on the MuJoCo-based LIBERO-Plus benchmark [ Fei et al., 2025 ] . The main comparison selects each configuration’s best evaluated checkpoint up to and including 13K steps by overall success, using the same evaluation set of 1,003 episodes across object, spatial, goal, and long-horizon suites.

This measures the downstream utility of the co-training data across simulator boundaries.

Experiments and Results — Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
Figure 7 : Generalization at the selected best-overall checkpoints. Each configuration is selected by its highest overall success up to and including 13K steps; all four suites use that selected checkpoint. Gains are largest for goals, followed by objects and spatial configurations; the long-horizon improvement is limited.

Sources and Demonstrations

Figures are reproduced from Team et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —
Co-Evolving Robot Orchestrators and Policies through Deployment
arXiv preprint

Co-Evolving Robot Orchestrators and Policies through Deployment

Xilun Zhang, Maggie Wang, Erik Bauer et al.

Vision-language-action (VLA) policies trained on large datasets are capable within their training domains, yet they still fail to generalize to the variety of situations a robot meets in real-world deployment.

vision vla reinforcement-learning learning-from-demonstration benchmark
PDF Intermediate