DroneVision AI

Architecture
ARM Platform Comparison
Accuracy and Performance
Optimisation Results
Concept

Inspiration

Autonomous drones commonly rely on GPS or onboard perception hardware for localization. Indoors and in GPS-denied environments, that adds weight, power consumption, computation and integration complexity.

DroneVision AI explores another approach: the room becomes the drone's eyes.

Four fixed cameras observe the flight area. DroneVision converts those views into a live 3D position, which can then be provided to PX4 as external vision for flight control. The perception workload stays off the aircraft and runs on Arm edge hardware.

For the Arm Create AI Optimization Challenge, we focused not only on making the localization work, but on understanding and optimizing the complete perception workload across different Arm architectures.

What it does

DroneVision AI is an external multi-camera localization pipeline for GPS-denied drone flight.

The processing chain contains five stages:

Image synchronization and freshness handling.
Per-camera perception using either a colour-marker reference detector or a YOLO26n learned detector.
Cross-camera association and tracking.
Calibrated multi-view DLT triangulation with outlier rejection.
State filtering and publication of the resulting 3D estimate.

The solution deliberately separates the reusable AI block from the simulator and flight-control environment.

The dronevision-ai repository contains the portable localization and benchmarking stack. The companion simulator repository contains Gazebo, four virtual cameras, PX4 SITL and the external-vision flight-control integration.

They communicate through network interfaces rather than sharing source code, allowing the same AI block to run on a development workstation, Raspberry Pi or NXP i.MX93.

Building a reproducible reference

Optimization is only useful if accuracy can be measured consistently.

We therefore created a committed corpus containing 337 synchronized four-camera frame sets with time-aligned 3D ground truth.

Using the deterministic colour-marker reference pipeline with the audited factory geometry and explicit detector-to-vehicle offset, DroneVision localizes:

337 / 337 frame sets
6.28 mm mean 3D error
6.19 mm median
10.87 mm P95
14.79 mm maximum

The colour detector is a reference rather than the final AI solution because it depends on a visible marker. It gives us a deterministic baseline against which the complete localization stack can be validated.

Optimizing the AI workload

Profiling the complete pipeline immediately showed where the optimization effort belonged: perception dominates the computational cost.

Replacing colour segmentation with YOLO turns the detection stage from a few milliseconds into the dominant embedded workload.

Instead of optimizing isolated kernels first, we benchmarked the complete system and tested optimization hypotheses experimentally.

Raspberry Pi 4

With FP32 ONNX Runtime inference on Cortex-A72:

Four-thread isolated inference: 93.7 ms
Four-camera threaded execution: approximately 319.9 ms per fix

Camera-level threading helped, but scaling was limited by shared memory bandwidth.

Multiprocessing was actually slower than threading because ONNX Runtime already releases Python's GIL during inference. Processes added frame serialization and memory traffic to a workload that was already memory-bandwidth constrained.

This changed our optimization strategy from more cores to moving fewer bytes and using more suitable execution hardware.

Raspberry Pi 5

On Cortex-A76, the same FP32 workload improved substantially:

Four-thread isolated inference: 37.36 ms
Best four-camera configuration: 114.50 ms per fix
Throughput: 8.73 fixes/s

Interestingly, camera-level parallelism provided much less benefit than on Raspberry Pi 4, demonstrating that the best execution strategy is architecture-dependent.

NXP i.MX93

CPU-only FP32 inference on the two Cortex-A55 application cores exposed a very different bottleneck:

Two-core isolated inference: 237.86 ms
Complete four-camera pipeline: 977.73 ms
Throughput: 1.02 fixes/s

The i.MX93, however, integrates an Arm Ethos-U65 NPU, which changed the optimization opportunity completely.

Making YOLO accelerator-friendly

The original YOLO export included post-processing operations that were poor candidates for complete NPU delegation.

We therefore changed the export to expose a raw [1, 5, 2100] prediction head and moved confidence filtering and NMS into shared CPU-side post-processing.

This also improved the FP32 reference:

Original end-to-end YOLO: 333/337, 36.42 mm mean 3D error
Raw-head YOLO: 337/337, 30.87 mm mean 3D error

We then performed full-integer quantization with INT8 input, activations and output, and compiled the graph using Arm Ethos-U Vela for ethos-u65-256.

The final model achieved 100% NPU graph delegation with no CPU fallback boundary nodes.

Final Arm result

On the NXP i.MX93:

Isolated neural-network inference

Cortex-A55 FP32: 237.86 ms
Ethos-U65 INT8: 37.43 ms
~6.35× faster

Complete four-camera localization loop

Cortex-A55 FP32: 977.73 ms / 1.02 fixes/s
Ethos-U65 INT8: 159.91 ms / 6.25 fixes/s
6.11× end-to-end speed-up

The model footprint also fell from 9.31 MB to 2.42 MB, a reduction of approximately 74%.

The accelerated INT8 path localized 336 of 337 frame sets. In the detailed NPU validation run it produced 52.39 mm mean 3D error, compared with 30.87 mm for the FP32 raw-head reference.

We report this accuracy cost alongside the performance gain rather than hiding it: for a physical AI system, speed and localization quality have to be evaluated together.

Experiments that did not work

Some of the most useful results came from optimizations that sounded attractive but failed under measurement.

ROI cropping

We tested Kalman-guided ROI cropping over multiple controlled motion profiles.

Although cropping reduced the number of source-image pixels dramatically, the exported ONNX graph still accepted a fixed 320 × 320 tensor. Every crop was resized to the same network shape, so the neural network performed essentially the same amount of work.

Fewer source pixels therefore did not translate into lower inference latency.

Frame skipping

Kalman-guided frame skipping increased throughput substantially, but fast, nonlinear and attitude-driven motion introduced significant localization error.

We therefore rejected a speed-only interpretation and retained both latency and accuracy as evaluation dimensions.

What we learned

The key lesson is that Arm edge-AI optimization is workload- and architecture-dependent.

The same model behaved very differently across Cortex-A72, Cortex-A76, Cortex-A55 and Ethos-U65.

Profiling changed several of our assumptions:

Python's GIL was not the inference bottleneck.
More processes could make a memory-bound workload slower.
Fewer source pixels did not reduce computation with a fixed-shape neural network.
Quantization became dramatically more valuable when paired with an appropriate dedicated accelerator.
Model export structure can determine whether an NPU can execute the complete graph.

Most importantly, we learned to measure the complete physical-AI workload, not only an isolated neural-network benchmark.

What's next

The next engineering steps include:

Quantization-aware training to recover INT8 localization accuracy.
Dynamic or genuinely smaller model shapes for useful ROI inference.
Improved NPU memory configuration.
Zero-copy / DMA-based data paths.
Expanded motion datasets beyond the primary hover corpus.
Motion-aware adaptive scheduling.
Closed-loop point-to-point autonomous flight.

DroneVision AI demonstrates a complete path from multi-camera perception to reproducible benchmarking and hardware-specific Arm acceleration — with both successful and unsuccessful optimizations documented as measured engineering results.