Stress-Query and Temporal Reliability of Vision-Language Models as copilots in Autonomous UAV Navigation

Hiago Sodre, Sebastian Barcelona, Vincent Sandin, Pablo Moraes, Andre S. Kelbouscas, Ricardo B. Grando · N/A · 2026

Summary

Integrating Vision-Language Models (VLMs) into autonomous UAV navigation enables high-level semantic reasoning but introduces critical temporal bottlenecks that can destabilize flight control.

Abstract Summary

Integrating Vision-Language Models (VLMs) into autonomous UAV navigation enables high-level semantic reasoning but introduces critical temporal bottlenecks that can destabilize flight control. This paper presents a systematic benchmark that evaluates the operational trade-offs among model scale, hardware constraints, and flight reliability. By subjecting the system to extensive stress tests on both a high-performance workstation and a constrained integrated-graphics laptop, we isolate the kinetic impact of inference latency against reasoning accuracy. Our findings reveal a performance dichotomy: lightweight models struggle with structural hallucinations that break deterministic state machines, while heavyweight models induce trajectory overcorrections due to inference lag. We demonstrate that a mid-weight architecture strikes the optimal balance, successfully rehabilitating navigation in degraded environments without compromising real-time stability across different hardware profiles.

Key Points

Integrating Vision-Language Models (VLMs) into autonomous UAV navigation enables high-level semantic reasoning but introduces critical temporal bottlenecks that can destabilize flight control.
This paper presents a systematic benchmark that evaluates the operational trade-offs among model scale, hardware constraints, and flight reliability.
By subjecting the system to extensive stress tests on both a high-performance workstation and a constrained integrated-graphics laptop, we isolate the kinetic impact of inference latency against reasoning accuracy.
Our findings reveal a performance dichotomy: lightweight models struggle with structural hallucinations that break deterministic state machines, while heavyweight models induce trajectory overcorrections due to inference lag.
We demonstrate that a mid-weight architecture strikes the optimal balance, successfully rehabilitating navigation in degraded environments without compromising real-time stability across different hardware profiles.

System Overview and Benchmarking Framework

To systematically evaluate the impact of Vision-Language Model scaling and cognitive latency on real-time autonomous flight, an exhaustive benchmarking framework was developed. The evaluations are executed within a fully physics-accurate Software-In-The-Loop (SITL) stack running PX4 Autopilot, ROS 2 Humble middleware, and Gazebo Harmonic (Figure 2 ).

The testbed operates under a decoupled hybrid control paradigm, deliberately separating low-level kinetic stability from high-level generative reasoning.

System Overview and Benchmarking Framework — Stress-Query and Temporal Reliability of Vision-Language Models as copilots in Autonomous UAV Navigation — Fig. 1: Proposed Hybrid FSM-VLM evaluation architecture. The deterministic FSM ensures strict real-time control, whilst the asynchronous VLM handles complex contextual anomalies.

Sources and Demonstrations

Figures are reproduced from Sodre et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.