Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models

Yiran Ling, Qing Lian, Jinghang Li, Qing Jiang, Tianming Zhang, Xiaoke Jiang, Chuanxiu Liu, Jie Liu, Lei Zhang · · 2026

Summary

Abstract Summary

In this paper, we propose GTA-VLA(Guide, Think, Act), an interactive Vision-Language-Action (VLA) framework that enables spatially steerable embodied reasoning by allowing users to guide robot policies with explicit visual cues. Existing VLA models learn a direct "Sense-to-Act" mapping from multimodal observations to robot actions. While effective within the training distribution, such tightly coupled policies are brittle under out-of-domain (OOD) shifts and difficult to correct when failures occur. Although recent embodied Chain-of-Thought (CoT) approaches expose intermediate reasoning, they still lack a mechanism for incorporating human spatial guidance, limiting their ability to resolve visual ambiguities or recover from mistakes. To address this gap, our framework allows users to optionally guide the policy with spatial priors, such as affordance points, boxes, and traces, which the subsequent reasoning process can directly condition on. Based on these inputs, the model generates a unified spatial-visual Chain-of-Thought that integrates external guidance with internal task planning, aligning human visual intent with autonomous decision-making. For practical deployment, we further couple the reasoning module with a lightweight reactive action head for efficient action execution. Extensive experiments demonstrate the effectiveness of our approach. On the in-domain SimplerEnv WidowX benchmark, our framework achieves a state-of-the-art 81.2% success rate. Under OOD visual shifts and spatial ambiguities, a single visual interaction substantially improves task success over existing methods, highlighting the value of interactive reasoning for failure recovery in embodied control. Details of the project can be found here: https://signalispupupu.github.io/GTA-VLA_ProjPage/

Key Points

Proposes Interactive Embodied Reasoning in Vision-Language-Action Models
Introduces a new dataset or benchmark
Provides open-source code or data

Methodology

Preliminaries. We formulate robotic manipulation as a conditional sequence modeling problem. At time step $t$, a standard Vision-Language-Action (VLA) policy $\pi$ receives multi-view RGB observations $\mathcal{I}_{t}=\{I_{t}^{(v)}\}_{v=1}^{V}$, a natural language instruction $L$, and the robot proprioceptive state $s_{t}$, and predicts a future action chunk where $k$ denotes the size of the action chunk.

Experiments

We evaluate our method along three main axes: standard benchmark performance, out-of-distribution (OOD) robustness, and the effectiveness of explicit visual guidance under spatial ambiguity. We first assess autonomous performance on established manipulation benchmarks, including LIBERO [ 23 ] and SimplerEnv [ 22 ] .

We then evaluate OOD generalization using our proposed SimplerEnv-Plus benchmark, which introduces systematic perturbations across visual, robot, language, and object-centric factors.

Experiments — Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models — Figure 3 : Interact-306K and automatic instruction annotation. Left: Dataset composition: 306K episodes collected from six manipulation sources ( e.g. , Bridge [ 30 ] , Fractal [ 38 ] , Droid [ 16 ] , and RoboMind variants [ 32 ] ). Right: Automatic annotation pipeline: keyframe extraction and task decomposition from trajectories, followed by open-vocabulary grounding and tracking to produce structured subtask instructions with temporally consistent object annotations.

More Implementation Details

To enable unified reasoning and action prediction, we serialize task instructions, intermediate reasoning steps, perception outputs, and low-level actions into a structured token sequence. For serialization and tokenization, the following example demonstrates how the training data is organized.

The sequence is composed of a set of special tokens that mark different semantic components, including task descriptions, object detections, manipulation targets, and action trajectories.

More Implementation Details — Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models — Figure from the paper. Source: Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models.

Real-World Deployment Details

Hardware Setting. We deploy our system on a single-arm AgileX Piper manipulator with one external Intel RealSense camera and one wrist-mounted RealSense camera. Robot observations are recorded at 30 FPS, and the manipulator is controlled in joint space. Inference is performed on a dual-GPU workstation with two NVIDIA RTX 5090 GPUs. Asynchronous Deployment.

We use an asynchronous deployment scheme in which the VLM reasoning branch and the Flow-Matching action head run on separate GPUs.

Real-World Deployment Details — Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models — Figure from the paper. Source: Guide, Think, Act: Interactive Embodied Reasoning in Vision-Language-Action Models.

Sources and Demonstrations

Figures are reproduced from Ling et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.