A Phenomenology of Neural Geometry in Vision-Language-Action Models

Mozes Jacobs, Aiden Swann, Mathilde Papillon, Siddharth Boppana, Fenil R. Doshi, Usha Bhalla, Thomas Icard, Matt Feiszli, Thomas McGrath, Vasudev Shyam, Demba Ba, Leon Bergen, Nina Miolane, Owen Lewis, Jack Merullo, Ekdeep Singh Lubana, Atticus Geiger, Matthew Kowal, Thomas Fel · N/A · 2026

Summary

Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it?

Abstract Summary

Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it? Prior work on vision-language-action (VLA) policies establishes that task-relevant variables are linearly readable from activations, yet readability certifies only that a variable is present and says nothing about the shape its values trace, which may be curved and multi-dimensional. Using controlled sweeps, natural rollouts, linear probes, and activation interventions, we build a taxonomy of seven task-relevant concepts in VLA models. We find that realized geometry is predominantly multi-dimensional and is shaped by the constraints under which the policy acts. Physical joint limits, for example, leave wrist orientation an open arc rather than a circle, and distance to the target, a single scalar, is realized in two orthogonal subspaces, a coarse ruler for the approach and a fine ruler for the grasp. The same geometries reappear in two further VLA models on real-world robot data, indicating that they are neither model-specific nor artifacts of simulation. We demonstrate how the findings are actionable by steering along the geometry a concept realizes. Overall, this work lays out an initial phenomenology of realized geometry in VLA policies and demonstrates that a concept's shape carries as much importance as its readability.

Key Points

Robotic manipulation is inherently geometric, but what shape does the world take inside the policy that performs it?
Prior work on vision-language-action (VLA) policies establishes that task-relevant variables are linearly readable from activations, yet readability certifies only that a variable is present and says nothing about the shape its values trace, which may be curved and multi-dimensional.
Using controlled sweeps, natural rollouts, linear probes, and activation interventions, we build a taxonomy of seven task-relevant concepts in VLA models.
We find that realized geometry is predominantly multi-dimensional and is shaped by the constraints under which the policy acts.
Physical joint limits, for example, leave wrist orientation an open arc rather than a circle, and distance to the target, a single scalar, is realized in two orthogonal subspaces, a coarse ruler for the approach and a fine ruler for the grasp.

Realization of Control Geometry

Before specifying the seven concepts of interest that this study will examine and documenting their realized geometry, we first clarify what is meant by realized and by geometry.

In particular, our aim here is to distinguish two questions that are frequently conflated in the literature: whether a concept can be recovered from a policy’s activations – which we term realized – and what geometric shape that recovered concept takes within activation space – which we term realized geometry .

Realization of Control Geometry — A Phenomenology of Neural Geometry in Vision-Language-Action Models — Figure 1: Control variables realize distinct geometries in a VLA policy. MolmoBot ( Deshpande et al., 2026 ) activations over many states (clouds) and controlled sweeps varying one physical quantity (traces). Clouds and traces are activations projected onto a low-dimensional basis (principal components for the gripper and wrist; the span of a linear probe’s weights for target position); for distance, the activations along the two ruler directions are plotted against the true distance (z axis). ( Gripper ) Open and closed states from natural rollouts separate along one axis; arrows show interventions reversing the command. ( Wrist ) Rotation sweeps pooled over 64 houses trace an open arc, not the circle unconstrained rotation would give. ( Target position ) A structured trajectory within the cloud of teleported target placements. ( Distance ) Two orthogonal rulers, coarse for approach and fine for grasp.

A Taxonomy of Realized Geometry in MolmoBot

We now build a taxonomy of how a model realizes the variables of pick and pick-and-place tasks. Our subject is MolmoBot ( Deshpande et al., 2026 ) , which pairs a VLM that reads the language instruction and camera images of the scene with an action expert that reads the robot’s proprioceptive state and generates action chunks via flow matching, cross-attending to the VLM at each layer (Appendix C.1 ).

Representations in Real Robots

All experiments in Section 3 were run in simulation. Since the gap between simulation and reality remains a central open problem in robotics ( Jakobi et al., 1995 ; Tobin et al., 2017 ; Kadian et al., 2020 ) , we ask whether the geometry we found carries over to real data.

We repeat our analyses on MolmoBot and two additional VLAs, $\pi_{0.5}$ ( Physical Intelligence et al., 2025b ) and MolmoAct2 ( Fang et al., 2026 ) , using episodes from the real-world DROID dataset ( Khazatsky et al., 2024 ) .

Representations in Real Robots — A Phenomenology of Neural Geometry in Vision-Language-Action Models — Figure 3: MolmoBot’s VLM encodes distance to the target in nearly orthogonal near- and far-range codes (layer 21). (a) Near (${<}15$ cm) and far (15–70 cm) probe $R^{2}$ after erasing the top-$k$ directions of the other probe, computed against each range’s own variance. Erasing far directions leaves near decoding intact while far decoding collapses, and vice versa. The intact ($k=0$) probes have errors of ${\sim}1.3$ cm (near) and ${\sim}3.6$ cm (far). (b) Near $R^{2}$ (${<}c$) vs. far $R^{2}$ (${\geq}c$) as the cutoff $c$ is swept. Clipped (‘round to c’) probes match probes fit separately on each range and outperform a single full-range probe. (c) Cosine between the near- and far-probe weight vectors as the cutoff $c$ is swept, for range-split and clipped (‘round to c’) pairs (fold mean $\pm$ s.d.). For $c\lesssim 30$ cm, both cosines stay below the cosine between two copies of the same probe fit on different halves of the data (dashed). On synthetic activations that encode distance along a single direction, the same probes are nearly parallel for $c\gtrsim 15$ cm (dotted). (d) Activations plotted by their meter- and centimeter-probe readouts, colored by true distance in meters (left) and centimeters (right). Far samples vary along the meter axis while the centimeter readout saturates, and near samples do the opposite, tracing an L.

Sources and Demonstrations

Figures are reproduced from Jacobs et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.