Underwater C3-JEPA: An Object-Centric Cross-View World Model for ROV Salvage

Yuncong Yang, Jinlong Li, Yulong Xue, Feng Wu, Chunwen Zhang, Lei Qiao, Xuyang Wang · N/A · 2026

Summary

We present Underwater C$^{3}$-JEPA (cross-view, control-conditioned, context-extended), an object-centric multi-view predictive world model for near-field heavy-load underwater ROV salvage.

Abstract Summary

We present Underwater C$^{3}$-JEPA (cross-view, control-conditioned, context-extended), an object-centric multi-view predictive world model for near-field heavy-load underwater ROV salvage. Without contact sensors, it predicts in latent space how the task-object state evolves through contact interaction and under the hydrodynamic lag of the vehicle, from synchronized multi-view RGB observations and vehicle control signals. C$^{3}$-JEPA encodes multi-camera observations into task-object and context tokens, fuses cross-camera evidence through held-out-view attention, and directly predicts future states conditioned on control. Weak binding anchors the target and gripper at low annotation cost, while SIGReg sharpens the geometric representation. Experiments show that the learned representation transfers substantially more task-relevant information to downstream probes than a reconstruction-free latent baseline, while keeping the predictor lightweight. The resulting predictive interface supports model-predictive-control (MPC) candidate evaluation and imagined-rollout behavior-agent training. Validation on real underwater video shows the same architecture recovering a withheld camera's object state and staying ahead of persistence, so the recipe transfers beyond simulation.

Key Points

We present Underwater C$^{3}$-JEPA (cross-view, control-conditioned, context-extended), an object-centric multi-view predictive world model for near-field heavy-load underwater ROV salvage.
Without contact sensors, it predicts in latent space how the task-object state evolves through contact interaction and under the hydrodynamic lag of the vehicle, from synchronized multi-view RGB observations and vehicle control signals.
C$^{3}$-JEPA encodes multi-camera observations into task-object and context tokens, fuses cross-camera evidence through held-out-view attention, and directly predicts future states conditioned on control.
Weak binding anchors the target and gripper at low annotation cost, while SIGReg sharpens the geometric representation.
Experiments show that the learned representation transfers substantially more task-relevant information to downstream probes than a reconstruction-free latent baseline, while keeping the predictor lightweight.

Method

Fig. 2 summarizes the pipeline. At time $t$, synchronized camera observations are $I_{t}^{1:V}$. A frozen DINOv3-L feature extractor [ 50 ] maps each view to patch features $X_{t}^{v}\in\mathbb{R}^{P\times d_{p}}$, and a temporal slot encoder returns $K$ slots $S_{t}^{v}\in\mathbb{R}^{K\times d_{s}}$. In simulation $V=4$; the architecture itself accepts the subset of views available on a platform.

Three binding slots represent the target UUV, the gripper, and the fixed ROV frame.

Method — Underwater C3-JEPA: An Object-Centric Cross-View World Model for ROV Salvage — Fig. 3: Binding-guided patch-to-slot grounding in C 3 -JEPA. Left to right: synchronized multi-view RGB frames from one real recording; grouping slot maps for the target (slot 0) and gripper (slot 1); the binding overlay, where weak anchors select patch support; the VideoSAUR-decoded slot masks that feed temporal state formation; and a PCA view of the patch features.

Experimental Setup and Evaluation

Table I lists the training configuration of the visual encoder and the control-conditioned predictor, shared by all experiments below. Data are collected in the ROS–Gazebo UUV Simulator environment [ 52 ] , which implements Fossen’s equations of motion for the vehicle dynamics [ 24 ] ; each recording provides synchronized cameras, vehicle state, gripper signals, poses, and event annotations. The caveat of Sec.

Results

The evidence below follows the evaluation ladder of Sec. IV . The learned representation preserves task-object semantics while reconstructing frozen visual features (Table II ). Across the recording-disjoint split, binding-guided training maintains high task-group Dice for the target UUV and gripper while held-out-view fusion lowers feature reconstruction error.

Binding guidance trades reconstruction for downstream utility.

Results — Underwater C3-JEPA: An Object-Centric Cross-View World Model for ROV Salvage — Fig. 5: UUV (slot 0) mask motion under the world model at two ranges: far ($\approx$$3.4$ m, 3 s rollout, top) and close ($\approx$$0.8$ m, recursive 5 s, bottom). Each panel pairs recorded (blue) and predicted (orange) mask-centroid tracks with the bottom-camera frame.

Field Validation on Real Underwater Video

All results above are trained and measured on simulated salvage scenes. The question here: does the same predictive state, with the same recipe, hold up on real underwater video? Everything here is trained from the raw field recordings alone; the simulator does not participate. Open-loop replay of recorded actions is a recognized evaluation protocol when closed-loop deployment is unavailable [ 58 ] .

Both recordings were collected in a test pool measuring 60 m by 20 m with a water depth of 10 m, under two operation modes (Fig.

Field Validation on Real Underwater Video — Underwater C3-JEPA: An Object-Centric Cross-View World Model for ROV Salvage — Fig. 6: Field validation. (a,b) Mask quality on the withheld camera for the mean-token, recovery and mask-term arms; dashed: that camera’s true tokens. (c) Token error of that recovery against the mean-token floor, over the seven slots (bars) and the target (rules). (d) Per-segment token error of a 36-step free rollout against persistence. (e) Error reduction over each reference. (f) Mask quality through the rollout.

Sources and Demonstrations

Figures are reproduced from Yang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.