CARLAverse: A Highly Modular, Distributed, and Multimodal Framework for Human-in-the-Loop Simulation

CARLAverse: A Highly Modular, Distributed, and Multimodal Framework for Human-in-the-Loop Simulation

|Authors: Patrick Rebling, Philipp Nenninger, Reiner Kriesten

|Venue: arXiv preprint | Year: 2026

|arXiv: 2609.11478v1

Abstract

The development of autonomous driving demands comprehensive testing in mixed-traffic scenarios involving vulnerable road users (VRUs), where purely artificial agents often fail to capture authentic human social negotiations. While human-in-the-loop (HITL) simulators enable safe investigation of these interactions, existing multi-agent platforms struggle with the network latency and synchronization constraints required for high-fidelity haptic feedback. To resolve this, we present CARLAverse, an open-source, multimodal simulation ecosystem. Extending modular hardware abstraction, CARLAverse integrates driving (DrivoCARLA), cycling (CycloCARLA), and pedestrian (WalkoCARLA) simulators into a shared virtual environment. Its core methodological contribution is a distributed physics architecture: latency-critical ego dynamics and high-frequency force feedback are computed locally on client nodes, while a central CARLA server orchestrates non-player character (NPC) physics and global traffic. By decoupling haptic control loops from network bottlenecks, CARLAverse enables scalable, cross-institutional HITL experiments without compromising physical immersion. Code and documentation: https://git.ieem-ka.de/simulator-environments/carlaverse

Key Contributions

• The development of autonomous driving demands comprehensive testing in mixed-traffic scenarios in…
• While human-in-the-loop (HITL) simulators enable safe investigation of these interactions, existi…
• To resolve this, we present CARLAverse, an open-source, multimodal simulation ecosystem
• Extending modular hardware abstraction, CARLAverse integrates driving (DrivoCARLA), cycling (Cycl…
• Its core methodological contribution is a distributed physics architecture: latency-critical ego …

Topics

• reinforcement-learning
• control
• tactile

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Rebling2026_260911478v1,
  title     = {CARLAverse: A Highly Modular, Distributed, and Multimodal Framework for Human-in-the-Loop Simulation},
  author    = {Patrick Rebling and Philipp Nenninger and Reiner Kriesten},
  year      = {2026},
  eprint    = {2609.11478v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2609.11478v1}
}