Robotics Research Hub
ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models

ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models

Gehan Zheng, Matthew Johnson-Roberson, Weiming Zhi · N/A · 2026

manipulation vision reinforcement-learning
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Contact-rich manipulation failures are often detected only after the robot has committed to contact.

Abstract Summary

Contact-rich manipulation failures are often detected only after the robot has committed to contact. This is especially limiting in wrist-camera setups: close gripper--object views help observe contact, but a poor approach may already push, miss, slip, or disturb the object before conventional detectors react. We introduce \emph{ContactGuard}, a pre-contact execution monitor for chunked visuomotor policies. Given the policy's planned action chunk, ContactGuard predicts its short-horizon consequence in latent visual space and aborts if the predicted future latent indicates likely failure. Its latent world model is trained from unlabelled robot trajectories to predict compact multi-view visual embeddings under planned actions, avoiding pixel-level video prediction. A lightweight failure probe is then trained from a small labelled set of pre-contact clips. At deployment, ContactGuard anchors prediction before an imminent contact event, rolls the model forward under the policy's own actions, and verifies the predicted post-contact latent. Across real-world contact-rich manipulation tasks, ContactGuard predicts failure more accurately than direct and corrupted-action ablations, and transfers to live robot as a pre-contact abort signal without modifying the underlying policy.

Key Points

  • Contact-rich manipulation failures are often detected only after the robot has committed to contact
  • This is especially limiting in wrist-camera setups: close gripper--object views help observe contact, but a poor approach may already push, miss, slip, or disturb the object before conventional detectors react.
  • We introduce \emph{ContactGuard}, a pre-contact execution monitor for chunked visuomotor policies
  • Given the policy's planned action chunk, ContactGuard predicts its short-horizon consequence in latent visual space and aborts if the predicted future latent indicates likely failure.
  • Its latent world model is trained from unlabelled robot trajectories to predict compact multi-view visual embeddings under planned actions, avoiding pixel-level video prediction.

Latent World Models for Pre-Contact Grasp Monitoring

We present ContactGuard , a pre-contact grasp monitor that predicts whether an imminent grasp is likely to fail before the gripper closes. It combines a JEPA-style latent world model with a frozen linear failure probe.

The world model is trained from unlabelled robot trajectories to predict action-conditioned future latents; the probe is trained from a smaller labelled grasp set to score the predicted post-contact latent.

Latent World Models for Pre-Contact Grasp Monitoring — ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
Figure 2: ContactGuard overview. A shared multi-view encoder maps camera observations to a latent context. An action-conditioned predictor rolls this context forward under the policy’s planned action chunk to produce future latents $\hat{z}_{t+1:t+K}$. A lightweight probe is trained offline on labelled pre-contact clips and used online to score the predicted post-contact latent $\hat{z}_{t+K}$. If the predicted failure probability exceeds threshold $\tau$, execution is aborted before contact. Training uses teacher-forced next-latent prediction; inference uses autoregressive rollout.

Experimental Results

We instantiate ContactGuard on grasp closure, a canonical short-horizon imminent-contact event with challenging visual geometry, occlusion, object variation, and a strict real-time pre-contact budget. We evaluate four grasp settings with ACT [ 30 ] as the underlying chunked visuomotor policy.

All metrics evaluate the predictor’s decision at the triggered pre-contact state, not the trigger itself or post-abort recovery. Our experiments ask four questions: Prediction quality.

Experimental Results — ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models
(a) Pick-and-place (cup)

Sources and Demonstrations

Figures are reproduced from Zheng et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced