End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking

End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking

Sitan Huang, Grace Gao · N/A · 2026

vision vla reinforcement-learning planning control learning-from-demonstration benchmark

Framework

N/A

License

N/A

Stars

N/A

Summary

Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances.

Abstract Summary

Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances. While modular navigation pipelines have long supported planetary rover navigation, robustness is compromised by measurement errors that propagate through hand-engineered components. End-to-end learned policies are an alternative approach to navigation, which maps observations directly to control actions without intermediate components. Vision-language-action models, in particular, have driven significant breakthroughs in robotics and autonomous vehicle research. By leveraging internet-scale vision-language pretraining, these models can be exploited for their commonsense reasoning and visual understanding to provide robust generalization with limited task-specific data. However, their applicability and performance have yet to be characterized for lunar surface operations. In this study, we design a vision-language-action policy for autonomous lunar rover docking and characterize its performance relative to a modular navigation pipeline. We develop a scalable training framework that generates safety-aware expert demonstrations in a high-fidelity lunar simulation environment. In closed-loop evaluations, our policy performs competitively relative to the modular baseline under perceptual disturbances unseen during training. To address the tradeoff between compute consumption and closed-loop safety, the policy adapts its execution horizon by determining how far the rover may safely travel in open loop. Adaptive-horizon execution reduces inference frequency while limiting hazard-contact rates relative to fixed horizons in the evaluated scenarios.

Key Points

  • Surface navigation on the Moon is subject to extreme illumination contrast and deployment-time perceptual disturbances.
  • While modular navigation pipelines have long supported planetary rover navigation, robustness is compromised by measurement errors that propagate through hand-engineered components.
  • End-to-end learned policies are an alternative approach to navigation, which maps observations directly to control actions without intermediate components.
  • Vision-language-action models, in particular, have driven significant breakthroughs in robotics and autonomous vehicle research.
  • By leveraging internet-scale vision-language pretraining, these models can be exploited for their commonsense reasoning and visual understanding to provide robust generalization with limited task-specific data.

Methodology

The lunar surface simulator LuPNT [ 3 ] , built on Unreal Engine, is used to generate photorealistic imagery and simulate closed-loop vehicle dynamics (Fig. 1(b) ).

Docking scenarios are 50 $\times$ 50 m regions sampled from a 5 m/px resolution digital elevation model, each containing 6.3 $\pm$ 4.1 (mean $\pm$ SD) boulders (diameter: 2.4 $\pm$ 0.8 m) and 5.0 $\pm$ 3.6 craters (diameter: 4.0 m; depth: 0.8 $\pm$ 0.1 m) on average. The sun elevation is sampled from 20 ∘ to 90 ∘ .

Methodology — End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking
(a) Schematic of the docking task and hazard landscape.

Evaluation

Unless otherwise specified, evaluations are conducted in $N=100$ procedurally generated scenarios using seeds unseen during training-data synthesis, with initial rover-to-landing-module separations of 30–50 m. The VLA policy uses a trajectory sampling temperature of 0.26 and adaptive-horizon execution unless otherwise noted.

The VLA policy and modular autonomy stack are evaluated under the following nominal and sensor-disturbance conditions (Fig. 4 ).

Evaluation — End-to-End Vision-Language-Action Policies for Autonomous Lunar Rover Docking
(b) The LuPNT [ 3 ] lunar surface simulation environment.

Sources and Demonstrations

Figures are reproduced from Huang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —
DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation
arXiv preprint

DexVerse: A Modular Benchmark for Multi-Task, Multi-Embodiment Dexterous Manipulation

Yunchao Yao, Zhuxiu Xu, Tianqi Zhang et al.

Building general-purpose dexterous manipulation policies requires benchmarks that go beyond isolated tasks to systematically evaluate policies across diverse interaction modes, sensory conditions, and robot embodiments.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
PDF Advanced