Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, Pieter Abbeel · OpenAI · 2017
Framework
MuJoCo / TensorFlow
License
MIT (OpenAI baseline)
Stars
0
Summary
Introduced domain randomization for vision-based manipulation: randomize simulation textures, lighting, and camera parameters so that the real world looks like just another variation.
Abstract Summary
Key Points
- First systematic introduction of visual domain randomization for sim-to-real transfer.
- Randomizes textures, lighting, camera pose, and colors instead of photorealism.
- Achieved zero-shot transfer to a real robotic arm for cube grasping.
- Inspired a decade of follow-up: dynamics randomization, shape DR, learning-to-simulate.
- Conceptually simple, reproducible, and applicable to any vision-based robotic policy.
Additional Notes
Implementation Tips
- Random camera pose within a small cone around the nominal viewpoint; extreme angles still help but slow convergence.
- Use procedural textures (Perlin noise, random colors) rather than curated datasets for best generalization.
- Combine with dynamics randomization for full sim-to-real robustness.
Related Papers
- Learning to Simulate (Ruiz et al., 2018)
- Sim-to-Real via Randomized Environments (Peng et al., 2018)
- CycleGAN-based Domain Adaptation (Bousmalis et al., 2018)
Related Papers
EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera
Christen Millerdurai, Shaoxiang Wang, Yaxu Xie et al. · SIGGRAPH 2026 · May 2026
Reconstructing the absolute 3D pose and shape of the hands from the user's viewpoint using a single head-mounted camera is crucial for practical egocentric interaction in AR/VR, telepresence, and hand-centric manipulation tasks, where sensing must...
Video
MOCHI: Motion Enhancement of Collaborative Human-object Interactions
Jiye Lee, Yonghun Choi, Jungdam Won · SIGGRAPH 2026 (Journal Track) · Jun 2026
Two-stage framework that enhances noisy multi-human object interaction (MHOI) data by generating plausible hand grasps and refining full-body motion via diffusion-based optimization.
DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation
Jusuk Lee, Seungjae Lee, Jonghun Shin et al. · arXiv preprint · May 2026
Robot manipulation critically depends on perception that preserves the action-relevant aspects of a scene. Yet most robot learning pipelines are built upon visual encoders pre-trained for static recog
GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization
Xiaosong Jia, Bowen Yang, Zuhao Ge et al. · RSS 2026 · May 2026
Vision-Language-Action (VLA) models aim for general robot learning by aligning action as a modality within powerful Vision-Language Models (VLMs). Existing VLAs rely on end-to-end supervision to implicitly enable the action decoding process to learn...