Robotics Research Hub
AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation

AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation

Featured

Wenxuan Guo, Xiuwei Xu, Yichen Liu, Xiangyu Li, Hang Yin, Huangxing Chen, Wenzhao Zheng, Jianjiang Feng, Jie Zhou, Jiwen Lu · Tsinghua University · 2026

vla foundation-models
PDF arXiv Code (29★)

Framework

N/A

License

N/A

Stars

29

Summary

AwareVLN introduces selective self-aware reasoning for VLN agents, triggering explicit spatial and progress analysis only at uncertain waypoints to improve robustness and explainability.

Abstract Summary

Vision-and-Language Navigation (VLN) agents typically predict actions end-to-end, lacking explicit awareness of their spatial state, task progress, or alignment with natural language instructions. AwareVLN introduces structured, self-aware reasoning that is selectively triggered at key navigation waypoints rather than consuming computation at every timestep. The model maintains an internal belief over task progress and spatial consistency, monitoring its own state for signs of uncertainty or misalignment with the instruction. When divergence is detected—such as entering the wrong room or missing a referenced landmark—AwareVLN invokes an explicit reasoning module that analyzes possible corrections and selects the most promising recovery action. This selective approach reduces unnecessary computation while improving navigation success rates. Evaluated on standard VLN benchmarks including R2R and REVERIE, AwareVLN achieves higher success and lower trajectory lengths than baseline end-to-end models. Importantly, the reasoning traces produced by AwareVLN are human-interpretable, making it easier to diagnose failure modes and build trust in embodied agents. The framework is compatible with existing VLN architectures and can be inserted as a lightweight reasoning head with minimal architectural changes.

Key Points

  • Selective self-aware reasoning for Vision-and-Language Navigation.
  • Only triggers explicit reasoning at uncertain waypoints, saving computation.
  • Maintains internal belief over task progress and spatial alignment.
  • Improves success rates on R2R and REVERIE benchmarks.
  • Produces human-interpretable reasoning traces for debugging.

Method

In this work, we consider vision-and-language navigation in unknown and continuous 3D environments (VLN-CE). The agent is provided with a natural-language instruction $\mathcal{I}=\{w_{1},\dots,w_{l}\}$, which describes how to navigate from the starting point to the destination, as illustrated in Figure 2 .

During execution, at each time step $t$, the agent receives an egocentric RGB observation stream $\mathcal{O}_{t}=\{\mathbf{x}_{0},\dots,\mathbf{x}_{t}\}$ captured by a monocular camera, without access to any depth or pose information.

Method — AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
Figure 2 : Framework of AwareVLN. (a) AwareVLN equips a unified vision-language model with both action prediction and self-reflective reasoning, allowing the agent to leverage past reasoning to guide future decisions. (b) The reasoning process is multi-dimensional and causal, sequentially describing the current scene, assessing progress, and planning the next step.(c) As illustrated by the BEV monitor, reasoning is sparsely and structurally triggered at key nodes, such as subtask boundaries.

Experiments

In this section, we first describe our experimental setting. Then we compare AwareVLN with state-of-the-art vision-and-language-navigation methods. We also conduct ablation experiments on our framework to validate the effectievness of each design choice. Finally, we provide qualitative analysis through rollout visualizations in both simulated and real-world environments.

For quantitative analysis, we compare different mainstream VLN methods on simulation benchmarks. Benchmarks.

Experiments — AwareVLN: Reasoning with Self-awareness for Vision-Language Navigation
Figure 3 : Automatic Data Engine. Key reasoning nodes are automatically identified using room-level semantics and ground-truth waypoints in the simulator, covering key events including subtask completion, path deviation, and incorrect stopping. For each key node, rich multimodal context is extracted and fed into a general VLM to automatically generate structured, causal reasoning supervision. This pipeline enables scalable, annotation-free construction of high-quality reasoning data.

Sources and Demonstrations

Figures are reproduced from Guo et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers