TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones

Summary

Abstract Summary

Nanodrones require accurate, real-time state estimation under severe sensing and computational constraints. We present TinyCVIO, a visual-inertial odometry system that co-designs miniature sensing, visual processing, and estimation for a commodity dual-core microcontroller with 520 kB SRAM. Lightweight LED constellations provide known geometry without surveyed positions or yaw angles, assuming placement on a common level plane. A streaming visual frontend tracks LED observations from a millimeter-scale camera at 29.2 FPS, while a rigid-board measurement model retains inter-LED constraints and streaming QR bounds estimation workspace for a fixed filter-state size. Across 19 hand-held hardware-in-the-loop datasets, the rigid-board model reduces mean absolute trajectory error by 27% relative to planar points. The complete system runs onboard a Crazyflie across nine flights at three speeds, achieving 3.5-3.7 cm mean absolute trajectory error and 0.50-0.60% relative pose error over 10 m segments, with mean estimate latency of 15.7-16.3 ms.

Key Points

Nanodrones require accurate, real-time state estimation under severe sensing and computational constraints.
We present TinyCVIO, a visual-inertial odometry system that co-designs miniature sensing, visual processing, and estimation for a commodity dual-core microcontroller with 520 kB SRAM.
Lightweight LED constellations provide known geometry without surveyed positions or yaw angles, assuming placement on a common level plane.
A streaming visual frontend tracks LED observations from a millimeter-scale camera at 29.2 FPS, while a rigid-board measurement model retains inter-LED constraints and streaming QR bounds estimation workspace for a fixed filter-state size.
Across 19 hand-held hardware-in-the-loop datasets, the rigid-board model reduces mean absolute trajectory error by 27% relative to planar points.

System Design

TinyCVIO combines a monocular camera and IMU with dual-core processing on a custom nanodrone PCB. Programmable I/O offloads camera readout and thresholding, while separate cores execute the visual frontend (Sec. IV ) and visual-inertial backend (Sec. V ), respectively (Fig. 2 ).

TinyCVIO uses an RP2354 dual-core Cortex-M33 microcontroller clocked at $300\text{\,}\mathrm{MHz}$, with $520\text{\,}\mathrm{kB}$ SRAM and $2\text{\,}\mathrm{MB}$ on-chip flash [ 18 ] .

System Design — TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones — Fig. 3: Visual frontend pipeline and example observations. (a) Spatial grouping and connected components, with an enlarged constellation. (b) Geometry-based LED identification, including candidate separation of a merged component. (c) Tracking preserves LED identities between frames using a fitted similarity transform; open circles denote previous positions and filled circles current tracked positions. Outputs contain image coordinates and board and LED identifiers.

Visual Frontend

The frontend converts threshold events into labeled LED observations for visual-inertial estimation (Fig. 3 ). Known constellation geometry and temporal tracking maintain correspondences as viewing distance changes: nearby constellations may fragment into separate spatial groups, while individual LED images may merge into a single connected component.

Threshold events arrive in raster order and are processed in a single pass.

Visual Frontend — TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones — Fig. 4: Frontend performance over six hand-held captures, binned by angular velocity, board distance, and image radius. Rows show localization error (mean, RMS, 5–95% band), mislabeled detections by type (dashed: aggregate), in-view boards excluding the tracker’s border exclusion zone that are detected (solid) and accepted by the backend (dashed), and board counts.

Visual-Inertial Backend

Our visual-inertial estimation framework is modeled after OpenVINS [ 14 ] , with a focus on embedded execution. Like OpenVINS, it separates the measurement model from propagation, nuisance elimination, measurement compression, and the filter update.

Within the Multi-State Constraint Kalman Filter (MSCKF) framework [ 15 ] , we present our streaming QR strategy and rigid-board measurement model, then describe sensor compensation and initialization before comparing independent-point models.

Visual-Inertial Backend — TinyCVIO: A Constellation-Aided Visual-Inertial Odometry System for Nanodrones — Fig. 5: Median ATE over $350$ runs per point, all axes log-scaled: (a) detection noise, with interquartile bands; (b) per-update LED budget at $0.5\text{\,}\mathrm{p}\mathrm{x}$ (solid) and $2\text{\,}\mathrm{p}\mathrm{x}$ (dashed) noise. Board budgets increase in steps of five LEDs. Free points diverge in 3–15% of runs; neither constrained model diverges.

Experimental Methodology

We combine controlled simulation, HITL end-to-end captures, and on-drone experiments to evaluate the proposed system. To isolate the effects of modeling assumptions, we generate trajectories in Webots [ 22 ] using a quadrotor model extended from CrazySim [ 23 ] , with the Crazyflie firmware’s estimator and controller. Figure-eight paths vary in lap period, amplitude, and altitude.

Results

We evaluate TinyCVIO in three stages: frontend detection and tracking on real hardware with Vicon ground truth, backend estimation in simulation, and the end-to-end system. We evaluate LED localization, labeling accuracy, and detection recall against the surveyed ground truth described in Sec. VI-C . Detected constellations have an overall LED localization RMS error of $0.6\text{\,}\mathrm{p}\mathrm{x}$ (Fig. 4 ).

Error varies little across the tested viewing distances and angular velocities.

Sources and Demonstrations

Figures are reproduced from Ozturk et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.