Human-in-the-Loop Geospatial Annotation for Rapid Dataset Construction in Field-Deployed UAV Systems

Human-in-the-Loop Geospatial Annotation for Rapid Dataset Construction in Field-Deployed UAV Systems

Morgan Masters, Nikolaas Bender, T. Luca Altaffer, Colleen Josephson, Steve McGuire · N/A · 2026

Framework

N/A

License

N/A

Stars

N/A

Summary

Real-world perception systems must adapt to changing environments, but manual image annotation cannot scale to field data volumes.

Abstract Summary

Real-world perception systems must adapt to changing environments, but manual image annotation cannot scale to field data volumes. We present BirdsEye, which shifts expert annotation from images to the field: an operator records target locations in world coordinates using RTK positioning and calibrated projective geometry propagates each observation to all frames where the target is visible. To quantify how well physical annotations align with image observations, we derive a first-order mapping from camera-pose uncertainty to pixel uncertainty and validate it against Monte Carlo simulation. This mapping is linear in the six per-axis pose variances, so it inverts into a sensor design tool: we give a sufficient condition converting an annotation tolerance into a convex set of admissible pose-noise budgets, a closed-form largest admissible scaling of a deployed sensor suite, and a unique per-axis pose specification under an equal-budget-share allocation. We also analyze the planar-surface approximation underlying the projection, which holds up to 10 degrees of terrain slope. By direct measurement, we show that system projection accuracy is sub-decimeter (sub-30 pixel) at AGL altitudes of 10-20m under conditions excluding sustained yawing. During an in-field case study across three agricultural sites, two field workers produced 12,524 annotated frames carrying 55,600 labels in roughly 12 hours (25.5x per-worker rate increase over manual labeling). Detectors trained on imagery collected by this workflow recovered 56-89% of in-view surveyed targets at a geographically distinct farm, at pre-registered operating points; human review of the leading configuration estimates detection precision at 83-87%, spanning three tie-break conventions for clusters carrying contradictory human verdicts.

Key Points

  • Real-world perception systems must adapt to changing environments, but manual image annotation cannot scale to field data volumes.
  • We present BirdsEye, which shifts expert annotation from images to the field: an operator records target locations in world coordinates using RTK positioning and calibrated projective geometry propagates each observation to all frames where the target is visible.
  • To quantify how well physical annotations align with image observations, we derive a first-order mapping from camera-pose uncertainty to pixel uncertainty and validate it against Monte Carlo simulation.
  • This mapping is linear in the six per-axis pose variances, so it inverts into a sensor design tool: we give a sufficient condition converting an annotation tolerance into a convex set of admissible pose-noise budgets, a closed-form largest admissible scaling of a deployed sensor suite, and a unique per-axis pose specification under an equal-budget-share allocation.
  • We also analyze the planar-surface approximation underlying the projection, which holds up to 10 degrees of terrain slope.

Materials and Methods

[node distance=1cm and 2cm] block = [rectangle, rounded corners, minimum width=3.2cm, minimum height=1cm, text centered, draw=black, fill=blue!20] \tikzstyle source = [rectangle, rounded corners, minimum width=3.5cm, minimum height=1cm, text centered, draw=black, fill=red!30] \tikzstyle sensor = [rectangle, rounded corners, minimum width=3.5cm, minimum height=1cm, text centered, draw=black, fill=green!30] \tikzstyle network = [rectangle, rounded corners, minimum width=3.5cm, minimum height=1cm, text centered, draw=black, fill=orange!30] arrow =...

Materials and Methods — Human-in-the-Loop Geospatial Annotation for Rapid Dataset Construction in Field-Deployed UAV Systems
Figure 3: An illustration of the variance within the general class of “burrow”. (Left) An obvious burrow with a small flattened mound. (Center) An occluded burrow and an obvious mound; is “mound” as important as “hole”? (Right) A very old, flattened mound with no visible hole; how degraded a target is allowable?

Results

To quantify system projective accuracy, we designed a validation procedure using geotagged AprilTag targets in a flat, level environment. This setup provides both pixel-space ground truth via AprilTag detections and world-frame ground truth via geotag RTK GPS positions, enabling comprehensive evaluation of both pixel and geospatial projection fidelity.

† The data of this table gives estimates of our effective GSD at both test altitudes: 2.219$\pm$0.138 mm @ 10 m; 4.376$\pm$0.021 mm @ 20 m.

Results — Human-in-the-Loop Geospatial Annotation for Rapid Dataset Construction in Field-Deployed UAV Systems
Figure 4: Illustration of the geometric relationship between a 3D geotag in the world and its 2D projection in the image. Also pictured are the position vectors relevant to the cross-products in the geotag detection method of Section 3.6.2 .

Sources and Demonstrations

Figures are reproduced from Masters et al. See the full paper for experimental details and the project page, when the paper links one, for demonstrations and videos.

Share

Related Papers

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies
arXiv preprint

A Sim-to-Real Integration Pipeline for Training and Deployment of Chunk-Based VLA Manipulation Policies

Mathilde Kappel, Clémence Grislain, Mohamed Chetouani et al.

Vision-Language-Action (VLA) models have become a prominent paradigm for mapping multimodal inputs, including semantic instructions, visual observations of the scene, and proprioceptive observations, to robot actions.

manipulation vision vla reinforcement-learning sim-to-real control learning-from-demonstration benchmark
PDF Advanced
Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You Think
arXiv preprint

Aim Short to Reach Far: Your Frozen World Model Can Plan Better Than You Think

Xvyuan Liu, Jianjie Fang, Chen Gao et al.

Planners built on visual world models commonly score each predicted outcome by its distance to the encoded goal image. We show that this target can limit control even with exact dynamics and globally optimal short-horizon search: reaching a goal may require actions that initially move away from it.

vision reinforcement-learning planning control human-robot-interaction benchmark
PDF Intermediate
Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?
arXiv preprint

Beyond Reconstruction: What Matters in Action Tokenization for Robot Policies?

Haoran Chen, Jingtian Ji, Samuel Wheeler et al.

Autoregressive action-token policies such as vision-language-action models require action tokenizers to translate discrete token sequences into precise control actions in continuous space. Many action tokenizers learn the mapping between tokens and actions via a reconstruction objective.

manipulation vision vla reinforcement-learning control benchmark
PDF Intermediate
Capability-Aware Arbitration for Semantic Intent-Based Shared Control
arXiv preprint

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

Zhaoda Du, Michael Bowman, Xiaoli Zhang

Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumption fails, high intent confidence can cause over-helping.

manipulation vision vla reinforcement-learning control learning-from-demonstration benchmark
Code PDF Intermediate
GitHub ★ —