Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving

Ruoyu Yao, Yusen Xie, Qingzhao Liu, Pei Liu, Zewei Yang, Yipeng Zhu, Xiaolong Wang, Jun Ma · N/A · 2026

Summary

Abstract Summary

Bridging the gap between the discrete reasoning of Vision-Language Models and the continuous, physics-constrained nature of autonomous driving remains a significant challenge. In this work, we introduce LaPla, a unified Vision-Language-Action (VLA) framework featuring latent-aligned planning to seamlessly ground semantic understanding in precise motion execution. We first design an action tokenizer based on a residual vector-quantized variational autoencoder (VQ-VAE), capturing vehicle kinematics and encoding trajectory features into a structured latent space. Rather than discrete codebook lookups that inevitably introduce quantization errors, LaPla repurposes this representation as a physical prior to bridge the modality gap between high-dimensional semantics and the raw action space. Specifically, given multimodal inputs integrating multi-view images, historical actions, and textual instructions, LaPla incorporates concurrent action queries to causally attend to the multimodal context in a single forward pass, projecting hidden states directly into the pretrained VQ-VAE latent space. The frozen decoder then translates these continuous latents into actions, effectively eliminating quantization errors and ensuring physically plausible trajectories while bypassing time-consuming autoregressive generation. Extensive experiments on the nuScenes benchmark demonstrate that LaPla achieves competitive open-loop performance, reducing long-horizon L2 error by 15.52% compared to state-of-the-art VLA methods. Closed-loop evaluations on the NVIDIA AlpaSim simulator further confirm its superior capability in ensuring smooth driving progress, improving the success rate by 33.34 percentage points with significantly reduced inference latency.

Key Points

Bridging the gap between the discrete reasoning of Vision-Language Models and the continuous, phy...
In this work, we introduce LaPla, a unified Vision-Language-Action (VLA) framework featuring late...
We first design an action tokenizer based on a residual vector-quantized variational autoencoder ...
Rather than discrete codebook lookups that inevitably introduce quantization errors, LaPla repurp...
Specifically, given multimodal inputs integrating multi-view images, historical actions, and text...

Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving

|Authors: Ruoyu Yao, Yusen Xie, Qingzhao Liu, Pei Liu, Zewei Yang, Yipeng Zhu, Xiaolong Wang, Jun Ma

|Venue: arXiv preprint | Year: 2026

|arXiv: 2609.04070v1

Abstract

Key Contributions

Bridging the gap between the discrete reasoning of Vision-Language Models and the continuous, phy…
In this work, we introduce LaPla, a unified Vision-Language-Action (VLA) framework featuring late…
We first design an action tokenizer based on a residual vector-quantized variational autoencoder …
Rather than discrete codebook lookups that inevitably introduce quantization errors, LaPla repurp…
Specifically, given multimodal inputs integrating multi-view images, historical actions, and text…

Topics

vision
vla
planning
benchmark

Code & Data

No code repository linked in paper metadata.

BibTeX

@article{Yao2026_260904070v1,
  title     = {Continuous Actions from Discrete Minds: Latent-Aligned Planning for End-to-End Autonomous Driving},
  author    = {Ruoyu Yao and Yusen Xie and Qingzhao Liu and Pei Liu and Zewei Yang and Yipeng Zhu and Xiaolong Wang and Jun Ma},
  year      = {2026},
  eprint    = {2609.04070v1},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url       = {https://arxiv.org/abs/2609.04070v1}
}