What Limits Vision-and-Language Navigation ?

Yunheng Wang, Yuetong Fang, Taowen Wang, Lusong Li, Kun Liu, Junzhe Xu, Zizhao Yuan, Yixiao Feng, Jiaxi Zhang, Wei Lu, Zecui Zeng, Renjing Xu · · 2026

Summary

Vision-and-Language Navigation (VLN) is a cornerstone of embodied intelligence.

Abstract Summary

Vision-and-Language Navigation (VLN) is a cornerstone of embodied intelligence. However, current agents often suffer from significant performance degradation when transitioning from simulation to real-world deployment, primarily due to perceptual instability (e.g., lighting variations and motion blur) and under-specified instructions. While existing methods attempt to bridge this gap by scaling up model size and training data, we argue that the bottleneck lies in the lack of robust spatial grounding and cross-domain priors. In this paper, we propose StereoNav, a robust Vision-Language-Action framework designed to enhance real-world navigation consistency. To address the inherent gap between synthetic training and physical execution, we introduce Target-Location Priors as a persistent bridge. These priors provide stable visual guidance that remains invariant across domains, effectively grounding the agent even when instructions are vague. Furthermore, to mitigate visual disturbances like motion blur and illumination shifts, StereoNav leverages stereo vision to construct a unified representation of semantics and geometry, enabling precise action prediction through enhanced depth awareness. Extensive experiments on R2R-CE and RxR-CE demonstrate that StereoNav achieves state-of-the-art egocentric RGB performance, with SR and SPL scores of 81.1% and 68.3%, and 67.5% and 52.0%, respectively, while using significantly fewer parameters and less training data than prior scaling-based approaches. More importantly, real-world robotic deployments confirm that StereoNav substantially improves navigation reliability in complex, unstructured environments. Project page: https://yunheng-wang.github.io/stereonav-public.github.io.

Key Points

Proposes What Limits Vision-and-Language Navigation ?
Provides open-source code or data

Methodology

In this section, we first define the task in Sec. 4.1 , and then introduce StereoNav . The proposed method alleviates instruction under-specification by incorporating precise or coarse target-point priors (Sec. 4.2 ), while leveraging stereo-based unified understanding modeling (Sec. 4.3 ) and joint prediction modeling (Sec. 4.4 ) to improve robustness against visual disturbances and achieve more accurate navigation.

We consider Vision-and-Language Navigation in Continuous 3D Environments.

Methodology — What Limits Vision-and-Language Navigation ? — Figure 4: Overview of StereoNav. StereoNav takes stereo RGB observations, a navigation instruction, and a target-location prior as input. The target prior is rendered as persistent visual guidance, while stereo observations are encoded into unified semantic, structural, and geometric tokens through 2D semantic, 2D structural, and 3D geometry encoders. These tokens are then processed by the MLLM for joint action and depth prediction. The right panels illustrate detailed designs of selected modules.

Experiment

We evaluate StereoNav from multiple perspectives. We first present the experimental setup in Sec. 5.1 ; then report comprehensive comparisons in simulated and real-world environments in Sec. 5.2 ; analyze robustness and reliability in Sec. 5.3 ; and provide ablation studies in Sec. 5.4 . Implementation Details. StereoNav is built upon InternVL-3.5-2B [ 36 ] .

Sources and Demonstrations

Figures are reproduced from Wang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.