SEVI: Safe Evidential Real-World Reinforcement Learning

At a glance

Problem: real-world RL pipelines learn contact-rich skills on hardware, but unsafe interactions during online learning block practical deployment. Source
Idea: an Evidential Safety Critic estimates future violation risk and epistemic uncertainty in one forward pass; high risk or weak evidence hands control to an offline-trained recovery policy. Source
Evidence: over three hardware runs on USB insertion, SEVI reached 0.83 success with 15 unsafe events, versus 0.68 success and 35 unsafe events for the SQRL recovery baseline. Source

Evaluation scope Evaluation covers two simulated MuJoCo manipulation tasks with five seeds each and one USB insertion task on a Franka arm, trained in three 3-hour hardware sessions per method. Source

SEVI on the real-world USB insertion task: when the task policy attempts an unsafe insertion, elevated risk or uncertainty triggers the recovery policy, which steers the robot to a safer region before handing control back. Original figure

How it works

The critic predicts a posterior predictive distribution over a temporal-difference risk target, giving expected future constraint-violation risk and epistemic uncertainty together. Source
The recovery policy is trained offline with TD3+BC on teleoperated recovery trajectories near failure boundaries: 150 per simulated task and 200 for the hardware task. Source
The task policy is trained with RLPD in the SERL framework, and SEVI leaves its reward, actor objective, critic update and replay sampling unchanged. Source
On hardware, safety labels come from manually marking the first failure timestep of each trajectory; every later timestep is labeled unsafe. Source

Evidence & results

Relative to SQRL, SEVI reduces unsafe events by 30.0% on Block Plucking, 13.4% on Transport Nut and 57.1% on USB Insertion, with the highest success rate on all three tasks. Evidence
SEVI has the lowest normalized detection time on Block Plucking and USB Insertion, with latency comparable to the point-estimate SQRL critic, while dropout and ensembles cost substantially more inference time. Evidence
Gating on risk plus uncertainty improved success and reduced unsafe events over risk-only gating under two different recovery behaviors on USB insertion. Evidence

Block Plucking example: an unsafe task action that could destabilize the stack triggers the recovery policy before the task resumes successfully. Original figure

Limitations & boundaries

The learned safety components give no formal guarantee of zero violations. Source
Recovery is not optimized for local or motion-efficient correction, and repeated task-recovery switching can occur near failure boundaries early in training. Source
The physical evaluation is a single task, USB insertion on one Franka Research 3 arm. Source

Takeaway

Our reading: adding an uncertainty check to a risk critic is a cheap safety layer over an unchanged RL learner, but it reduces unsafe events rather than guaranteeing that none occur. Source

Resources

Full text ↗PDF ↗

Figures: the paper's authors. Editorial source and image review: 2026-10-26. Findings are author-reported, not independently reproduced by Robotics Research Hub.

Original abstract

Real-world reinforcement learning has enabled contact-rich manipulation tasks to be trained directly on physical robots, but unsafe interactions during online learning remain a key barrier to practical deployment. We propose Safe Evidential Real-World Reinforcement Learning (SEVI), a framework for reliability-aware safety intervention during real-world policy learning. SEVI introduces an Evidential Safety Critic that estimates both future constraint-violation risk and epistemic uncertainty in a single forward pass. During online learning, SEVI evaluates each task-policy action before execution; if the action is predicted to be unsafe or the prediction is insufficiently supported by evidence, SEVI overrides it with an offline-trained recovery policy that guides the robot toward a lower-risk region. We evaluate SEVI on two simulated manipulation tasks and a real-world USB insertion task under in-distribution and distribution-shifted conditions. Across task-learning and safety-critic baselines, SEVI achieves the strongest trajectory-level safety prediction, the highest final task success, and the lowest unsafe-event counts. SEVI also reduces unsafe events by 13.4 to 57.1 percent relative to recovery-based safe RL while preserving low-latency execution-time filtering. The project page is available at: https://sevi-safe.github.io/

Citation

@article{Shithil2026_261031829,
  title     = {SEVI: Safe Evidential Real-World Reinforcement Learning},
  author    = {Shaekh Mohammad Shithil and Lantao Liu},
  year      = {2026},
  eprint    = {2610.31829},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2610.31829}
}