Intrinsic Robot Rewarding: Reusing VLA Representations for Autonomous Evaluation and Policy Improvement

Position and motivation

Vision–language–action models connect visual observations and language instructions to robot actions. RT-2 demonstrated the transfer of web-derived semantic knowledge into robotic control, while OpenVLA made a broadly pretrained VLA available for downstream adaptation [ 1 , 2 ] . These developments provide an increasingly capable foundation for manipulation.

Adapting such a policy to an industrial task also produces a valuable local resource with demonstrations that show both, how the task is performed and what successful execution looks like.

Related research and proposed contribution

OpenVLA combines DINOv2 and SigLIP visual features with a language backbone and action prediction [ 2 , 5 , 6 ] . Its optimized fine-tuning recipe, OpenVLA-OFT, uses parallel decoding, action chunking, continuous actions, and an L1 regression objective [ 7 ] . This provides a practical base policy for studying internal outcome evaluation.

A residual RL controller can build on its regression-based action interface while supplying the stochastic learning mechanism for subsequent improvement.

Available TRL 4 demonstrator and experimental foundation

An operational VLA demonstrator is available at TRL 4, providing a laboratory-validated experimental foundation for IRR. The demonstrator comprises a COMAU Racer 3 six-axis industrial arm with a Robotiq Hand-E gripper, a third-person USB camera, VR teleoperation, and a ROS control stack [ 4 ] . The baseline study uses 245 demonstrations of placing a small green cube into a container.

Images, proprioceptive observations, and end-effector delta commands were recorded at a nominal 5 Hz.

Proposed IRR formulation

Let $g$ be a task with a written success specification, $o_{t}$ denote its camera observation, $p_{t}$ the robot’s proprioceptive state, and $\pi_{0}$ the imitation-trained VLA policy. Let $\phi$ be a frozen checkpoint of its visual encoder and $P$ a documented feature extraction and pooling operation.

For a single image, define The encoder layer, image preprocessing, token pooling, and normalization are part of the method and must be fixed before evaluation. DINOv2 features, SigLIP features, and their fusion will be compared.

Research questions and decisive experiments

RQ1 – How effectively do existing policy representations support task evaluation? We hypothesize that a task-conditioned reference bank can distinguish successful endpoints from realistic near misses with a low false-positive rate. The evaluation will include correct placement, objects beside the container, objects still held above it, wrong-object placement, premature release, and partial occlusion.

Evaluation design and evidence standards

The first task reproduces the documented cube-to-container experiment. Two planned extensions introduce target-container placement with separation and placement into a fixture with visible pose constraints. These are laboratory tasks motivated by industrial handling.

Natural-product sorting and precision assembly provide subsequent application directions, with additional acceptance measurements wherever appearance alone is inadequate.

Scientific value and industrial relevance

IRR advances a practical research position, that representations and demonstrations used to establish robot behavior can also support its evaluation and improvement. Reuse can extend the function of an existing VLA deployment while keeping the reward mechanism compact. The core design introduces task references and a scoring operation, with no separate learned evaluator or additional perception backbone.

This makes the approach straightforward to relate to the robot’s existing data and model configuration.

Sources and Demonstrations

Figures are reproduced from Schaffer et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.