D3DWA: Adaptive Weight and Prediction-Horizon for Dynamic Window Approach via Dueling Double Deep Q-Network

Summary

The Dynamic Window Approach (DWA) is widely used for local navigation, but its performance depends strongly on parameters that are typically fixed before navigation.

Abstract Summary

The Dynamic Window Approach (DWA) is widely used for local navigation, but its performance depends strongly on parameters that are typically fixed before navigation. In particular, the appropriate prediction horizon can vary with local free space: longer horizons support efficient motion in open areas, whereas shorter horizons help preserve feasible motions in narrow or cluttered regions. This paper proposes D3DWA, an adaptive DWA framework based on a Dueling Double Deep Q-Network (D3QN), which jointly selects the DWA evaluation weights and prediction horizon from a continuous navigation state at every control step while retaining DWA's trajectory generation and collision checking. In eight simulated environments, including unseen layouts, D3DWA reached every goal. Real-robot experiments further showed that D3DWA completed all three tested configurations, including a constrained case in which the weights-only variant timed out. These results demonstrate the benefit of jointly adapting the evaluation weights and prediction horizon. Additional material is available at https://mertcookimg.github.io/d3dwa/

Key Points

The Dynamic Window Approach (DWA) is widely used for local navigation, but its performance depends strongly on parameters that are typically fixed before navigation.
In particular, the appropriate prediction horizon can vary with local free space: longer horizons support efficient motion in open areas, whereas shorter horizons help preserve feasible motions in narrow or cluttered regions.
This paper proposes D3DWA, an adaptive DWA framework based on a Dueling Double Deep Q-Network (D3QN), which jointly selects the DWA evaluation weights and prediction horizon from a continuous navigation state at every control step while retaining DWA's trajectory generation and collision checking.
In eight simulated environments, including unseen layouts, D3DWA reached every goal
Real-robot experiments further showed that D3DWA completed all three tested configurations, including a constrained case in which the weights-only variant timed out.

Method

At every D3DWA control step a body-frame observation is built from the lidar scan and wheel odometry; a D3QN agent maps it to a discrete action; the action decodes to a DWA parameter tuple and is published to the planner; and the DWA planner uses that tuple to score and execute a velocity command. Fig. 2 shows the discussed architecture.

DWA forms the dynamic window from the current velocity and the acceleration limits, and draws a grid of $N_{v}\times N_{\omega}$ samples $(v,\omega)$ from it.

Method — D3DWA: Adaptive Weight and Prediction-Horizon for Dynamic Window Approach via Dueling Double Deep Q-Network — Fig. 4 : Executed trajectories of DWA III ($\{w_{p},w_{c},w_{s}\}=\{2,1,1\}$), coloured by elapsed time. The robot starts at the origin and the goal is marked in red.

Simulation

The simulation system was implemented using the Robot Operating System (ROS) and Gazebo. The parameters used for the planner and reinforcement-learning agent are summarized in Table II . Six methods were compared.

Three fixed-parameter DWA configurations were used following the baseline setting: DWA I $\{w_{p},w_{c},w_{s}\}=\{1,1,2\}$, DWA II $\{1,2,1\}$, and DWA III $\{2,1,1\}$, all with a fixed prediction horizon of $\tau=4$ s. DQDWA [ 2 ] was used as the tabular Q-learning baseline.

Simulation — D3DWA: Adaptive Weight and Prediction-Horizon for Dynamic Window Approach via Dueling Double Deep Q-Network — Fig. 5 : Executed trajectories of the tabular baseline DQDWA [ 2 ] , coloured by elapsed time. The robot starts at the origin and the goal is marked in red.

Real-World Experiments

The proposed method was evaluated on a physical Kachaka mobile robot in the same room used to construct the simulation environments, as shown in Fig. 7 . Three static-obstacle configurations, denoted as Cases R1–R3, were used to evaluate navigation performance under different spatial arrangements of obstacles. The obstacle layouts and executed trajectories for the three configurations are shown in Figs. 8 – 10 .

Case R1 places five obstacles between the start position and a goal at $(3.0,0.0)$.

Real-World Experiments — D3DWA: Adaptive Weight and Prediction-Horizon for Dynamic Window Approach via Dueling Double Deep Q-Network — Fig. 6 : Executed trajectories of D3DWA. Top row (a)–(h): coloured by elapsed time. Bottom row (i)–(p): the same trajectories coloured by the prediction horizon $\tau$ selected by the agent. The robot starts at the origin and the goal is marked in red.

Sources and Demonstrations

Figures are reproduced from Jooyandeh et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.