Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning

Ismail Geles, Leonard Bauersfeld, Markus Wulfmeier, Davide Scaramuzza · University of Zurich, Google DeepMind · 2026

Summary

The first demonstration of superhuman, safe, multi-agent drone racing using a MARL framework trained in simulation and transferred to real Crazyflie nano-drones.

Abstract Summary

Autonomous drone racing has long struggled to match human-level performance in multi-agent competitive scenarios because single-agent policies implicitly assume a static world. This paper presents a multi-agent reinforcement learning (MARL) framework that trains racing drones to anticipate opponent trajectories and proactively choose safe, aggressive racing lines. The key contribution is a safety-conditioned action space in which each drone predicts the probability of collisions with nearby agents and selects actions that maximize race progress while keeping collision probability below a tunable threshold. The policy is trained entirely in simulation on a photorealistic racing track using centralized training with decentralized execution (CTDE), allowing agents to share experiences during training but act independently at test time. Remarkably, the policy transfers directly to real Crazyflie nano-drones without any real-world fine-tuning. In extensive outdoor experiments, the drones achieve lap times faster than the human world champion while maintaining zero mid-race collisions. This is the first demonstration of superhuman, safe, multi-agent drone racing and validates the hypothesis that MARL, when combined with careful safety conditioning, can produce policies that are both faster and safer than human experts.

Key Points

First demonstration of superhuman safe multi-agent drone racing.
Multi-agent RL trained in simulation transfers to real Crazyflie drones.
Safety-conditioned action space balances aggression and collision avoidance.
Outperforms human world champion lap times with zero collisions.
Uses photorealistic sim-to-real transfer for outdoor deployment.

Results

We evaluate our approach in both real-world and simulated multi-agent settings. We use a single policy trained with four agents that generalizes to varying numbers of opponents. In physical experiments, we race with up to 4 competitors, including a champion-level human pilot. The policy maintains both safety and competitive performance across configurations, from solo time trials to dense four-agent races.

Materials and Methods

We train and evaluate policies in simulation using Flightmare [ 36 ] integrated with the Agilicious framework [ 13 ] , which provides high-fidelity dynamics for large-scale reinforcement learning. We extended the simulator to support multiple quadrotors simultaneously, enabling collision detection and aerodynamic interactions between agents.

Multi-agent training is parallelized across 144 environments, each simulating four quadrotors, to accelerate data collection while exposing agents to varied multi-agent interaction dynamics.

Materials and Methods — Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning — Figure 2 : A , Average race completion through self-evaluation with identical policies from solo to 8-player races. Each data point represents four policies per method, each completing 64 races with varied starting positions. Shaded regions indicate the average of standard deviations per policy. B , Crash rates by collision type (gate, wall, opponent) from the races completed in A. C , Sample races illustrating behavioral differences. Single-agent policies collide with 85% probability at the first gate in four-player races. The independent PPO example shows agents from the same training run achieving collision-free racing through implicit coordination; this behavior does not generalize, dropping to approximately 60% race completion in self-evaluation. League-play trains against diverse opponents, producing strategies that remain robust across agent configurations.

Supplemetary Material

Table S1 lists the coefficients used in the reward function defined in the Materials and Methods section. The progress coefficient $\lambda_{1}$ provides the primary learning signal, while the remaining coefficients are scaled to balance competing objectives without dominating the progress reward.

The relatively small values for ranking ($\lambda_{3}$) and opponent proximity ($\lambda_{4}$) encourage competitive behavior and safe distancing without overriding the fundamental gate-progress objective.

Supplemetary Material — Superhuman Safe and Agile Racing through Multi-Agent Reinforcement Learning — Figure 3 : A , Learned value function visualized by varying the ego agent position in the $(X,Z)$ plane while fixing opponent positions. Trajectories are from a real-world overtaking maneuver at the Split-S gate, where competitors pass through an upper gate followed by a lower gate. The ego agent (orange) overtakes opponent 2 (red) across the four time steps shown. Low-value regions near opponents reflect learned collision avoidance, while high values indicate regions of greater expected reward. For positions beyond the next gate, the visualization assumes the gate has been passed to avoid artificially low values from the agent seeking to return. B , Corresponding real-world trajectories showing the ego agent overtaking opponent 2 while avoiding opponent 1. C , Particle-based downwash simulation for a quadrotor moving in the positive $X$ direction. This aerodynamic interaction model is integrated into the training simulator, allowing agents to experience the physical effects of nearby competitors.

Sources and Demonstrations

Figures are reproduced from Geles et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.