Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms

Yanchen Guan, Xingcheng Liu, Bin Rao, Chengyue Wang, Guofa Li, Yunjian Li, Lishengsa Yue, Zhiyong Cui, Chengzhong Xu, Zhenning Li · N/A · 2026

Summary

Abstract Summary

End-to-end autonomous driving has evolved from camera-to-control regression toward planning-oriented systems that use structured representations, trajectory-level outputs, and increasingly realistic evaluation protocols. This survey reviews this transition across behavior cloning, conditional imitation learning, privileged distillation, BEV and vectorized planning, unified perception-prediction-planning architectures, world-model-based planners, and vision-language-action systems. We argue that the key distinction in modern end-to-end driving is not whether intermediate representations are used, but whether they are learned, supervised, and evaluated to support safe, feasible, and route-compliant planning. To organize the literature, we synthesize existing methods along four axes: input representation, planning output, supervision signal, and evaluation protocol. We further examine the benchmark shift from open-loop trajectory matching to closed-loop simulation, non-reactive real-log evaluation, long-tail testing, and human-preference-aware metrics. Our analysis highlights that architectural progress is difficult to interpret without benchmark-consistent evaluation, and that displacement-based open-loop metrics alone provide limited evidence for safe and human-aligned driving. We conclude with open challenges in uncertainty-aware planning, learner-expert mismatch, runtime safety assurance, language-action grounding, world-model validation, and reproducible benchmarking.

Key Points

End-to-end autonomous driving has evolved from camera-to-control regression toward planning-oriented systems that use structured representations, trajectory-level outputs, and increasingly realistic evaluation protocols.
This survey reviews this transition across behavior cloning, conditional imitation learning, privileged distillation, BEV and vectorized planning, unified perception-prediction-planning architectures, world-model-based planners, and vision-language-action systems.
We argue that the key distinction in modern end-to-end driving is not whether intermediate representations are used, but whether they are learned, supervised, and evaluated to support safe, feasible, and route-compliant planning.
To organize the literature, we synthesize existing methods along four axes: input representation, planning output, supervision signal, and evaluation protocol.
We further examine the benchmark shift from open-loop trajectory matching to closed-loop simulation, non-reactive real-log evaluation, long-tail testing, and human-preference-aware metrics.

Scope, Methodology, and Related Surveys

Recent surveys have reviewed end-to-end driving, vision-language driving, world models, foundation models, and autonomous-driving datasets from different perspectives [ 23 , 7 , 24 , 25 , 26 , 27 , 28 , 29 ] .

This survey is motivated not by the absence of prior summaries, but by the need to connect three issues that are increasingly inseparable in planning-oriented E2E-AD: model architecture, evaluation protocol, and emerging foundation-model-based driving paradigms.

Scope, Methodology, and Related Surveys — Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms — Fig. 1: Search-and-screen workflow used for this survey. The workflow records the main sources, search terms, citation tracing, screening criteria, and final categorization of the reviewed literature.

Evolution Toward Planning-Oriented E2E-AD

Figure 2 summarizes the main historical trend reviewed in this section. The evolution of E2E-AD is not a single architectural lineage, but a shift from imitating low-level control toward learning representations, supervision signals, and evaluation protocols that make planning behavior more inspectable and benchmarkable. The roots of E2E driving predate modern deep learning.

Planning-Oriented Taxonomy

Figure 3 summarizes the taxonomy used in this survey. We categorize planning-oriented E2E-AD methods along four axes: input representation, output space, supervision signal, and evaluation protocol. Table II makes these axes explicit. This taxonomy avoids a common limitation of classifying E2E-AD methods only by backbone names. A transformer encoder, diffusion decoder, or VLM is not by itself a driving formulation.

Architectures and Learning Paradigms

Direct control prediction is attractive because it minimizes the number of engineering components between perception and actuation. Yet it gives the network little room to express uncertainty or to separate strategic planning from actuator-level control. Waypoint and trajectory prediction provide a more useful intermediate action space [ 88 ] .

A waypoint sequence can be supervised from logs, converted to control by a controller, and checked for collisions, drivable-area violations, or comfort [ 89 ] .

Architectures and Learning Paradigms — Planning-Oriented End-to-End Autonomous Driving: Architectures, Evaluation, and Emerging Paradigms — Fig. 4: Why open-loop and closed-loop evaluations can disagree. Open-loop trajectory matching tests similarity under the logged state distribution, while closed-loop evaluation tests the policy under the states it actually induces.

World Models and VLA Driving

Planning is about consequences. A purely reactive policy can imitate expert actions, but it does not explicitly ask what will happen if the ego vehicle accelerates, yields, nudges forward, or changes lane. World models attempt to fill this gap by learning latent dynamics of the scene.

As detailed in Table VI , GenAD framed autonomous driving as generalized predictive modeling and introduced OpenDV-2K as a driving video resource [ 110 ] . Driving into the Future connected multi-view visual forecasting with planning [ 111 ] .

Benchmarks and Metrics

Datasets such as BDD100K, nuScenes, Waymo Open Dataset, and Waymo Open Motion Dataset remain foundational resources for perception, tracking, forecasting, and planning-related tasks [ 116 , 68 , 117 , 118 ] . However, a dataset is not automatically a driving benchmark. A planning benchmark must define an input interface, an output interface, a scenario distribution, metrics, and comparison rules.

Safety, Robustness, and Deployability

E2E systems are often criticized as black boxes. The stronger version of this critique is not that all learned systems are uninterpretable, but that their failure modes are difficult to localize when the entire stack is optimized jointly. Recent work addresses this through interpretable intermediate structure, causal analysis, and safety constraints. Safety-critical evaluation settings are summarized in Table X .

InterFuser uses interpretable sensor fusion and safety-enhanced action constraints [ 57 ] .

Reproducibility and Future Directions

Beyond benchmark scores, the long-term value of planning-oriented E2E-AD depends on whether methods can be reproduced, compared, and stress-tested under aligned protocols. This section summarizes reporting requirements, public-resource levels, and open challenges for future research. Table XI lists items that should be extracted from papers whenever possible.

If a paper does not report an item, it should be marked as unavailable. Not all public resources support the same level of reproducibility.

Sources and Demonstrations

Figures are reproduced from Guan et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.