RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms

Xi Chen, Fengqi Hu, Zeyu Shen, Jiaying Xu, Shangding Gu, Pan Xu, Anqi Liu, Laixi Shi · N/A · 2026

Summary

Robust reinforcement learning (RL) is critical for reliable real-world deployment, where policies may fail under diverse shifts and disruptions.

Abstract Summary

Robust reinforcement learning (RL) is critical for reliable real-world deployment, where policies may fail under diverse shifts and disruptions. Existing robust RL algorithms offer a rich set of mechanisms and principles, yet two major barriers limit their broader potential: limited algorithm reusability and an unclear empirical capability boundary. We introduce RobustRLlib, an algorithm-centric library and benchmark that addresses both challenges. It integrates 17 representative robust online, offline, and safe RL algorithms under a unified interface for substantially improved reusability, together with an enhanced evaluation platform spanning diverse real-world shifts and advanced humanoid and vision-language-action tasks. Systematic evaluation over 273 perturbation configurations provides an up-to-date capability map of current robust RL algorithms, revealing where they remain effective and where substantial robustness gaps persist. As an open-source, user-friendly platform, RobustRLlib offers a reproducible and extensible codebase for applying existing methods to new problems and developing new robust RL algorithms.

Key Points

Robust reinforcement learning (RL) is critical for reliable real-world deployment, where policies may fail under diverse shifts and disruptions.
Existing robust RL algorithms offer a rich set of mechanisms and principles, yet two major barriers limit their broader potential: limited algorithm reusability and an unclear empirical capability boundary.
We introduce RobustRLlib, an algorithm-centric library and benchmark that addresses both challenges
It integrates 17 representative robust online, offline, and safe RL algorithms under a unified interface for substantially improved reusability, together with an enhanced evaluation platform spanning diverse real-world shifts and advanced humanoid and vision-language-action tasks.
Systematic evaluation over 273 perturbation configurations provides an up-to-date capability map of current robust RL algorithms, revealing where they remain effective and where substantial robustness gaps persist.

RobustRLlib : An Algorithm-Centric Robust RL Benchmark

RobustRLlib is organized around algorithm attribution : it records how a robust RL method changes learning, which shift the method claims to address, and which base algorithm realizes the method. The disruptor toolbox then instantiates the claimed target shift independently of the base learner, while the evaluation platform measures both common robustness and method-specific specialization.

Experimental Evaluation

The evaluation proceeds in three blocks over our library. Section 4.1 measures how much robust methods gain across the whole library and how that gain behaves as the shifts intensify. Section 4.2 isolates each shift source and composes them, to ask which shift source a method actually defends. Section 4.3 carries the same interface to deployment-oriented humanoid and vision–language–action (VLA) systems.

The library implements 23 algorithms: 6 standard RL references, 15 robust RL and 2 robust safe RL algorithms.

Experimental Evaluation — RobustRLlib: A Unified Library and Benchmark for Robust Reinforcement Learning Algorithms — (a) MDP-element shifts.

Sources and Demonstrations

Figures are reproduced from Chen et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.