Robotics Research Hub
VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan

Ole Hoffmann, Mateo de Mayo, Daniel Cremers · N/A · 2026

vision planning
PDF arXiv

Framework

N/A

License

N/A

Stars

N/A

Summary

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner.

Abstract Summary

Perception in robotics and XR fundamentally relies on good state estimation. Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner. Efficiency in these systems allows for smaller, cooler, and lighter devices. GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets. However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor. We instead leverage the vendor-agnostic Vulkan API, originally designed for the strict performance requirements of 3D graphics applications. In this work, we present VkVIO, the first, to the best of our knowledge, cross-platform GPU-accelerated VIO method. We provide state-of-the-art accuracy with causal estimates required for real-time operation. We deploy VkVIO on a diverse range of devices spanning a workstation, a laptop, and an extremely inexpensive single-board computer, while outperforming CUDA-based systems on the same hardware. VkVIO enables possibilities for low-latency, low-power, and low-cost VIO in robotics and XR.

Key Points

  • Perception in robotics and XR fundamentally relies on good state estimation
  • Visual-inertial odometry (VIO) and Simultaneous Localization and Mapping (VI-SLAM) are proven ways of achieving this goal in a cost-effective and accurate manner.
  • Efficiency in these systems allows for smaller, cooler, and lighter devices
  • GPU acceleration is a natural approach for reducing latency, thanks to their wide availability in platforms like embedded computers, mobile phones, and XR headsets.
  • However, previous works in the literature have limited themselves to the use of CUDA for this task, significantly reducing deployment options to a single vendor.

Method

Vulkan is a cross-platform API for graphics and compute on modern GPUs. Its explicit programming model gives applications direct control over resource management, synchronization, and command submission, enabling high performance across many platforms [ 12 ] . Compute work is issued as dispatch commands, which are recorded into a command buffer and submitted to a queue for execution.

Each dispatch runs the compute pipeline bound at recording time and launches groupCountX $\times$ groupCountY $\times$ groupCountZ workgroups.

Method — VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan
Fig. 2: Data flow of the VkVIO frontend for one frame. The CPU uploads the stereo images, the previous keypoints $\mathrm{KP}_{t-1}$, and the mask buffer. The four stages run as compute shaders from a single command buffer. The outputs $\mathrm{TR}_{t}$ (tracked and cross-matched keypoint positions) and $\mathrm{DET}_{t}$ (newly detected keypoints) are downloaded back to the CPU.

Results

For dataset selection, we prioritize a broad selection of devices over sequence count, to showcase the versatility of VkVIO and how its performance scales with input resolution. We utilize visual-inertial datasets from the SLAM literature centered around real-time robotics and XR. Fig.

5 shows an overview of the type of input data provided by each device in the datasets, with the pixel area of each device drawn to scale.

Results — VkVIO: Cross-platform GPU Acceleration for Visual-Inertial Odometry with Vulkan
Fig. 3: GPU feature detection, from image to invocation. Each $24\times 24$ grid cell (blue) maps to one workgroup, its pixels to subgroups of 32 invocations (pink), and each pixel (black) to one invocation, which runs the four-point early-out, the FAST-9 segment test, and $3\times 3$ NMS. A bitonic sort per workgroup keeps the $k$ strongest survivors, appended to $\mathrm{DET}_{t}$.

Sources and Demonstrations

Figures are reproduced from Hoffmann et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

AcousticDiffusion: Semantically Conditioned Audio-Guided Diffusion Policy for Search-and-Rescue Assistance
arXiv preprint

AcousticDiffusion: Semantically Conditioned Audio-Guided Diffusion Policy for Search-and-Rescue Assistance

Iana Zhura, Didar Seyidov, Dmitrii Plotnikov et al.

Navigating toward human callers is an important capability for rescue robots operating where visual contact is degraded or occluded. We present AcousticDiffusion, a semantically conditioned, audio-guided diffusion policy for human-directed navigation.

locomotion vision reinforcement-learning planning
PDF Advanced