FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems

Songyuan Zhang, Baljeet Singh, Sarthak Ranjeet Kaingade, Chuchu Fan, Bryan Trinh · N/A · 2026

Summary

Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates.

Abstract Summary

Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates. Safety filters separate them at action execution, but classical designs require an analytic safety function and dynamics model, and standard minimal-intervention filters are myopic to long-horizon task return because they minimize only instantaneous action deviation. Hard projections are also undefined when no safe action exists. We present FAITH, a feasibility-aware, model-free framework that approximates the optimal state-action safety value and amortizes minimal-intervention filtering with a feedforward network. The task policy optimizes the task return through the filtered dynamics, which recovers the feasible constrained problem without a competing safety term in the task-policy update. When no action satisfies the learned safety condition, the same filter approaches the action with minimum predicted peak harm. On a double integrator example and a Safety Gym environment, FAITH achieves the highest return among methods with no feasible-start violations and matches the lowest harm from infeasible starts. On a 29-DoF humanoid, it reaches a 99.95% safety rate while retaining 97% of the unfiltered return in Walking-Avoid, and obtains the highest measured safety rate in Push-Avoid by learning to sacrifice balancing and fall away from the protected region. The same policies are also demonstrated on a real-world Unitree G1 humanoid.

Key Points

Safe reinforcement learning commonly places safety and task performance in the same policy objective, where they can introduce competing updates.
Safety filters separate them at action execution, but classical designs require an analytic safety function and dynamics model, and standard minimal-intervention filters are myopic to long-horizon task return because they minimize only instantaneous action deviation.
Hard projections are also undefined when no safe action exists
We present FAITH, a feasibility-aware, model-free framework that approximates the optimal state-action safety value and amortizes minimal-intervention filtering with a feedforward network.
The task policy optimizes the task return through the filtered dynamics, which recovers the feasible constrained problem without a competing safety term in the task-policy update.

Problem Setting and Preliminaries

We consider a discrete-time system $s^{k+1}=f(s^{k},a^{k})$ with state $s\in\mathcal{S}$, action $a\in\mathcal{A}$, and unknown deterministic dynamics $f$. A reward $r:\mathcal{S}\times\mathcal{A}\to\mathbb{R}$ specifies the task. For a policy $\pi:\mathcal{S}\to\mathcal{A}$, define its discounted return as 1 1 1 Trajectories under $\pi$ satisfy $s^{k+1}=f(s^{k},\pi(s^{k}))$.

We omit these dynamics when they are clear from context. where $\gamma\in[0,1)$ and the return is assumed finite.

Problem Setting and Preliminaries — FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems — Fig. 1: FAITH separates safety learning from task-return optimization. (a) The task policy proposes a nominal action, and the safety filter produces the executed action. The dashed box denotes the filtered dynamics seen by the task policy . (b) The safety critic , backup actor , and safety filter learn from replay. The backup actor supplies actions for the safety critic’s Bellman target, while critic gradients guide the backup actor and safety filter . (c) PPO updates the task policy using nominal-action likelihood ratios and filtered-rollout returns. Only the task policy and safety filter are evaluated for deployment.

Method

To address these limitations, we propose FAITH . FAITH approximates the optimal safety critic with a backup actor supplying low-harm actions for the Bellman target. A learned filter uses this critic to approximate the solution to ( 5 ) when the learned safety condition is feasible and to prioritize lower harm otherwise.

The task policy is trained from filtered rollouts using only reward, so safety does not appear as a competing objective in policy updates.

Method — FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems — Fig. 2: Double integrator results: safety rate and task return of feasible starts, and mean peak harm of infeasible starts. Left: For feasible starts, FAITH achieves a $100\%$ safety rate, and the highest return among all safe methods (error bars show min-max ranges). Right: For infeasible starts, FAITH achieves the lowest peak harm ($\pm$ shows the standard deviation).

Simulation Experiments

We conduct experiments to answer three research questions. (Q1): Starting from feasible states, how much task return can FAITH retain while respecting the safety constraint? (Q2): When harm is unavoidable, can FAITH minimize the harm? (Q3): Does FAITH scale to a high-dimensional system with whole-body constraints?

We compare against four representative ways of enforcing safety in the literature: RCRL [ 6 ] , a primal-dual feasibility-aware method that enforces a policy-conditioned safety value in the objective.

Simulation Experiments — FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems — Fig. 3: Trajectories in the double integrator environment. FAITH matches the feasible set closely in visited states. From feasible starts ($\circ$), FAITH rides the feasibility boundary to achieve the highest return and stays safe. When infeasible ($\triangle$), FAITH applies maximum braking to minimize peak harm.

Hardware Experiments

We conduct qualitative hardware demonstrations on a $29$-DoF Unitree G1, with boxes representing the protected region ( ). In the pipeline ( Fig. 6 ), $10$ Hz LiDAR scans undergo SLAM-based ground removal [ 42 ] . We retain the nearest point in each of $64$ equal-width azimuth bins covering $360^{\circ}$, matching the simulation observation format.

At $50$ Hz, FAITH combines this observation (held between scans) with the current robot state to generate joint-position targets. A $1000$ Hz PD controller then tracks these targets.

Hardware Experiments — FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems — Fig. 4: Car-Circle Results. With all initial states being feasible, FAITH has $100\%$ safety and high task return (error bars show min-max ranges).

Sources and Demonstrations

Figures are reproduced from Zhang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.