AcousticDiffusion: Semantically Conditioned Audio-Guided Diffusion Policy for Search-and-Rescue Assistance

Iana Zhura, Didar Seyidov, Dmitrii Plotnikov, Hajira Amjad, Miguel Altamirano Cabrera, Dzmitry Tsetserukou · N/A · 2026

Summary

Abstract Summary

Navigating toward human callers is an important capability for rescue robots operating where visual contact is degraded or occluded. We present AcousticDiffusion, a semantically conditioned, audio-guided diffusion policy for human-directed navigation. A frozen pretrained audio recognizer processes 10.24 s windows, with speech gating and distress-aware prioritization converting recognition outputs into source-level navigation roles. Microphone-array direction-of-arrival measurements are recursively integrated into a robot-centric Bayesian bird's-eye-view belief field. Ego-motion compensation aligns successive observations, progressively constraining source position while preserving bearing-induced range uncertainty. The semantic belief, recent acoustic observations, audio features, and robot state condition a diffusion model that generates waypoint trajectories. On a synthetic-navigation validation set using recorded audio, AcousticDiffusion achieves a mean end-point bearing error of 11.20 degrees, with 91.78% of trajectories aligned within 30 degrees of the caller. Distractor rejection ranges from 89.20% to 98.99%, and the policy favors a HELP-designated caller over a competing speaker in 91.07% of windows. Deployed online on a ZSL-1 quadruped without additional retraining, it achieves a mean bearing error of 64.9 degrees, compared with 98.2 degrees for A* and 90.4 degrees for RRT, with a mean planner compute time of 6.07 ms. Despite imperfect acoustic localization, the reported mean final source distance is reduced from 3.96 m for the classical planners using ODAS-derived (Open embedded Audition System) guidance to 2.48 m, a 37.4% improvement. These results demonstrate the framework's ability to translate uncertain acoustic observations into closer approaches to human callers.

Key Points

Navigating toward human callers is an important capability for rescue robots operating where visual contact is degraded or occluded.
We present AcousticDiffusion, a semantically conditioned, audio-guided diffusion policy for human-directed navigation.
A frozen pretrained audio recognizer processes 10.24 s windows, with speech gating and distress-aware prioritization converting recognition outputs into source-level navigation roles.
Microphone-array direction-of-arrival measurements are recursively integrated into a robot-centric Bayesian bird's-eye-view belief field.
Ego-motion compensation aligns successive observations, progressively constraining source position while preserving bearing-induced range uncertainty.

Method

AcousticDiffusion generates trajectories toward human sound sources using frozen audio recognition, a recursive Bayesian bird’s-eye-view (BEV) field, and a conditional diffusion policy (Fig. 2 ). The system receives ODAS-separated audio, source-bearing observations, and robot odometry.

Speech determines whether a source attracts the robot; distress provides a preference among speakers rather than a prerequisite for approach. The default configuration uses acoustic conditioning without a visual encoder.

Method — AcousticDiffusion: Semantically Conditioned Audio-Guided Diffusion Policy for Search-and-Rescue Assistance — Fig. 2 : Overview of AcousticDiffusion. Pretrained audio recognition provides semantic predictions and audio features. Sound-source bearings and odometry update a Bayesian BEV belief. Fused conditioning guides a 1-D U-Net that generates trajectories from Gaussian noise through 16 DDIM iterations. The BEV output and denoising stages are illustrative.

EXPERIMENTS AND RESULTS

Speech recordings are drawn from LibriSpeech [ 19 ] , while emotional speech is obtained from RAVDESS [ 20 ] . Fearful speech serves as a proxy for distress; these acted utterances are not literal requests for assistance. Audio is processed in 10.24 s windows, with shorter recordings padded. Successive clips are concatenated without overlapping crossfades.

Synthetic scenes combine a designated caller, competing speakers, and non-speech distractors. A potential-field expert provides trajectory supervision.

EXPERIMENTS AND RESULTS — AcousticDiffusion: Semantically Conditioned Audio-Guided Diffusion Policy for Search-and-Rescue Assistance — Fig. 3 : Qualitative visualizations from the AcousticDiffusion deployment on the quadruped robot. Both panels show the microphone-array frame ( respeaker_link ), the displayed trajectory, the learned source-head diagnostic (magenta), and the BEV attraction-field estimate (red), overlaid on the local map. Panel (b) additionally shows the ODAS bearing observation in yellow.

REAL-ROBOT EVALUATION

Experiments used a ZSL-1 quadruped equipped with an NVIDIA Jetson Orin NX, a Livox MID-360 LiDAR/IMU, and a 16-kHz Seeed reSpeaker XVF3800 four-microphone array, with ODAS 2.1 supplying direction-of-arrival tracks and separated audio. The array centre sits $0.25$ m behind and $0.30$ m above base_link , rotated $180^{\circ}$ in yaw. All perception, inference, and navigation ran on-board, with no external compute.

Localization fused motor odometry with LiDAR–IMU measurements via planar NDT-OMP scan matching against a prior point-cloud map.

Sources and Demonstrations

Figures are reproduced from Zhura et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.