X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching

Tianyu Yang, Yiming Zeng, Wenzhe Cai, Yuqiang Yang, Jiaqi Peng, Hui Cheng, Jiangmiao Pang, Tai Wang · N/A · 2026

Summary

Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle planner suited to a single nominal robot.

Abstract Summary

Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle planner suited to a single nominal robot. This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations. Post-training the policy with reinforcement learning (RL) offers a principled remedy. However, previous RL for diffusion approaches lead to only marginal improvements. This is because the intractable likelihood of diffusion policies renders policy gradients unstable in addition to inefficient policy exploration. To address these challenges, we propose a data-efficient diffusion RL post-training framework - GQRM (Group Q-score Reweighted Matching). Our framework introduces two complementary designs: (i) a self-bootstrapped exploration strategy with behavior perturbation that preserves the pretrained policy prior, and (ii) a group Q-score normalization mechanism that computes per-trajectory values on each state for efficient reweighted score matching. By conducting distributed online RL training across heterogeneous embodiments, the resulting fine-tuned policy, X-NavDP, achieves state-of-the-art cross-embodiment visual navigation performance, improving the overall success rate from 61.20% to 84.28% in simulation and 10% to 65% in real-world hard cases. The code and model are publicly available at https://yty-sky.github.io/x-navdp-project-page.

Key Points

Pretraining navigation diffusion policies rely on large-scale expert demonstrations
These data are typically generated by a fully-informed oracle planner suited to a single nominal robot.
This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations.
Post-training the policy with reinforcement learning (RL) offers a principled remedy
However, previous RL for diffusion approaches lead to only marginal improvements

Preliminaries

Following the prior works [ 30 , 5 ] , we formulate visual navigation as a conditional trajectory generation problem. Given an observation $\mathbf{o}_{t}$ consisting of RGB-D images and a goal specification $\mathbf{g}$, the policy generates an action chunk $a^{0}=\{(x_{i},y_{i})\}_{i=1}^{H}$ containing $H$ future waypoints through iterative denoising.

The forward diffusion process adds Gaussian noise over $K$ steps: where $\beta_{k}$ is the noise schedule.

Preliminaries — X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching — Figure 2: Overview of the X-NavDP pipeline. (a) We adopt the NavDP backbone and inject embodiment information via FiLM layers. (b) A structured trajectory perturbation scheme mixes goal-conditioned and goal-agnostic samples for efficient exploration (denoted as pointgoal and nogoal in the figure, respectively). (c) Within-group $Q$-score normalization produces adaptive weights for reweighted score matching. (d) The framework enables large-scale cross-embodiment RL.

Method

X-NavDP is an RL post-trained diffusion policy for navigation, built upon the pretrained NavDP [ 5 ] . Fig. 2 shows the overall pipeline. Given local visual observations and navigation goals, our policy outputs an action chunk with $H$-step future waypoints.

Our core objective is to enhance policy performance through online interactive simulation experience, while preserving the generalization ability of the pretrained model.

Method — X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching — Figure 3: Qualitative trajectory comparison. X-NavDP produces recovery, detour, and safer behaviors that are absent from the NavDP policy. The visualization is provided with VGGT-$\Omega$ [ 33 ] .

Experiments

We design experiments to answer four questions: (1) Does X-NavDP outperform strong navigation baselines across different embodiments and scenes? (2) Does RL post-training induce new navigation behaviors such as recovery and replanning? (3) How much does each proposed component contribute to the final performance? (4) What failure modes remain?

We build our simulation benchmark in IsaacLab [ 18 ] using scenes from GRScenes-100 [ 43 , 8 ] . For RL post-training, we use 56 training scenes, including 47 home scenes and 9 commercial scenes.

Sources and Demonstrations

Figures are reproduced from Yang et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.