FARM: Reading Failure Signals from the Internal Predictive States of a Frozen Robotic World Model

Haoran Pei, Mingrui Luo, Senbao Wang, Haoran Lv, Jie Guo, Sheng Zhong, Ruixi Ci · N/A · 2026

Summary

Reliable robot deployment requires online failure monitoring, yet existing monitors mainly derive risk from proxy signals or train dedicated monitoring components.

Abstract Summary

Reliable robot deployment requires online failure monitoring, yet existing monitors mainly derive risk from proxy signals or train dedicated monitoring components. We ask whether the internal predictive states of a frozen pretrained robotic world model already contain directly decodable failure information. Failure-Aware Readout from World Models (FARM) trains only a 33,985-parameter supervised readout over frozen VLA-JEPA predictive states, producing step-wise failure scores and causal trajectory risk. Five-fold out-of-fold evaluation across seven source tasks reaches 85.68/88.59 pooled AUROC/AUPRC, and FARM gives the best Seen performance among 15 matched baselines on the 10-task benchmark. Across four real-robot populations on PIPER X, SO-101, and Franka, fixed-readout transfer and readout-only adaptation test deployment shifts without updating the predictive backbone. FARM also discriminates failures from partial causal histories and adds 0.2256 ms mean CUDA latency once the frozen state is available. These results support frozen predictive world-model states as reusable features for causal, transferable, and low-overhead execution monitoring.

Key Points

Reliable robot deployment requires online failure monitoring, yet existing monitors mainly derive risk from proxy signals or train dedicated monitoring components.
We ask whether the internal predictive states of a frozen pretrained robotic world model already contain directly decodable failure information.
Failure-Aware Readout from World Models (FARM) trains only a 33,985-parameter supervised readout over frozen VLA-JEPA predictive states, producing step-wise failure scores and causal trajectory risk.
Five-fold out-of-fold evaluation across seven source tasks reaches 85.68/88.59 pooled AUROC/AUPRC, and FARM gives the best Seen performance among 15 matched baselines on the 10-task benchmark.
Across four real-robot populations on PIPER X, SO-101, and Franka, fixed-readout transfer and readout-only adaptation test deployment shifts without updating the predictive backbone.

Failure-Aware Readout from World Models

At each replanning step, FARM compresses the WM-state tokens into a 32-dimensional failure-aware representation and produces a step-wise score. The following formulation defines the readout, its training objective, and causal risk aggregation (Fig. 1 ). Let $\mathcal{D}=\{\tau_{i}\}_{i=1}^{M}$ be a set of robot trajectories.

Trajectory $\tau_{i}$ belongs to task $k_{i}$ and has outcome $y_{i}\in\{0,1\}$, where 1 denotes Failure and 0 denotes Success.

Failure-Aware Readout from World Models — FARM: Reading Failure Signals from the Internal Predictive States of a Frozen Robotic World Model — Fig. 1: FARM reads the WM state $H_{t}^{WM}\in\mathbb{R}^{768\times 1024}$. A shared token projection, attention pooling, and lightweight MLP produce $z_{t}^{F}\in\mathbb{R}^{32}$ and the step-wise failure score $s_{t}$; $q_{t}=\max_{\tau\leq t}s_{\tau}$ aggregates only observed scores into causal trajectory risk.

Experiments

We evaluate FARM through four questions aligned with the paper’s claim. Q1: Representation. Do frozen WM states contain failure-relevant information beyond coarse hidden-state statistics? Q2: Detection. Can the 33,985-parameter readout convert that information into a competitive multi-task detector? Q3: Transfer.

How does the source-trained readout behave on new policies, robots, and visual domains, and can residual target shift be absorbed by readout-only adaptation? Q4: Online use.

Experiments — FARM: Reading Failure Signals from the Internal Predictive States of a Frozen Robotic World Model — Fig. 2: Training-task scaling on the fixed 446-trajectory FARM-Expanded OOF population. Curves connect the nested 1-, 3-, 5-, and 7-task source-training settings; markers report (AUROC, AUPRC).

Sources and Demonstrations

Figures are reproduced from Pei et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.