CORAL: Curriculum-Optimized Reward Adaptation for LiDAR-Based Goal-Directed Urban Driving

Summary

Reinforcement learning is promising for autonomous urban driving, but long-horizon goal-directed navigation asks a policy to acquire several competing behaviors at once--reaching a distant goal, tracking a route, avoiding obstacles, obeying signals--and a fixed objective gives no order in which to learn them.

Abstract Summary

Key Points

• Reinforcement learning is promising for autonomous urban driving, but long-horizon goal-directed navigation asks a policy to acquire several competing behaviors at once--reaching a distant goal, tracking a route, avoiding obstacles, obeying signals--and a fixed objective gives no order in which to learn them.
• This paper presents CORAL, which advances two schedules together: a five-stage curriculum that progressively lengthens routes and tightens behavioral constraints, and a stage-aware reward whose component weights shift emphasis from mission progress toward route following, safety, smoothness, and rule compliance as the task hardens.
• The policy is a multi-stream actor-critic network trained with Proximal Policy Optimization (PPO) in CARLA on a compact 99-dimensional state pairing a polar LiDAR histogram with vehicle telemetry, ego-frame route geometry, and traffic-rule indicators--no point-cloud encoder, no bird's-eye-view rasterization.
• Against two PPO baselines under an identical protocol, CORAL reaches the goal in all twenty evaluation episodes on the longest routes under the full set of behavioral constraints, where the baselines reach 5% and 10%; a factorial ablation shows that neither schedule alone matches their combination: removing either lowers both success and route completion, and disabling both drops success to 55%.
• Trained in one town, the policy transfers zero-shot to seven unseen towns, succeeding in 68-98% of episodes on routes of the same 100-150 m length, with mean lateral deviation below 0.35 m.

Methodology

This section presents the proposed curriculum-guided reinforcement learning framework for goal-directed autonomous driving. We formulate goal-directed urban driving as a continuous-control reinforcement learning problem in which, at each timestep, the agent maps an observation of the driving scene to steering, throttle, and brake commands, and is trained in the CARLA simulator using PPO. Fig.

1 gives an overview of CORAL; this subsection walks through the end-to-end pipeline and points to the subsection that develops each component in detail.

Experiments and Results

This section first describes the experimental setup—the simulation environment, training configuration, evaluation protocol and metrics, and baselines—and then reports the comparative and generalization results, followed by two ablation studies.

Simulation environment: All experiments use the CARLA simulator (v0.9.15) on a goal-directed navigation task, in which the agent drives from a randomly sampled start to a target destination while following a planned route and obeying traffic signals.

Sources and Demonstrations

Figures are reproduced from Saleem et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.