Inspiration
In many real-world edge applications—ranging from agricultural crop health monitoring to automated waste sorting in municipal facilities—cloud-dependent computer vision systems suffer from high latency, expensive bandwidth, and network instability. CortexVision was born out of the need for a privacy-first, zero-cloud computer vision engine that processes visual data at the point of capture. The core objective was to prove that complex, real-time object detection can be achieved directly on low-power Arm Cortex-A architectures without relying on continuous cloud connectivity or expensive, power-hungry discrete GPUs.
What it does
On-Device Visual Inference: Processes live video feeds locally on Arm Cortex-A processors (e.g., Raspberry Pi 4/5 or NVIDIA Jetson) using lightweight quantized neural networks ($\text{INT8}$).Hardware-Accelerated Execution: Direct integration with Arm NN and CMSIS-NN primitives leverages Arm NEON SIMD vector registers to accelerate tensor operations and matrix multiplications at the hardware level.Low-Latency Asynchronous Pipeline: Decouples video ingestion from neural network execution using multi-threaded frame buffering, sustaining consistent frame rates ($30+\text{ FPS}$) without CPU thermal throttling or frame drops.Privacy-First & Offline Functionality: Delivers 100% offline analysis with zero cloud dependency, eliminating external API latency, recurring bandwidth costs, and data exposure risks.
How we built it
Model Architecture & Pre-processing: A lightweight convolutional neural network (CNN) was trained in PyTorch for specialized vision tasks. The input video stream is decoded and downsampled asynchronously using an OpenCV frame-buffer pipeline.Quantization & Graph Compilation: To maximize hardware throughput, the full-precision ($\text{FP32}$) weights and activations were quantized to signed 8-bit integers ($\text{INT8}$). Floating-point values $q$ were mapped to quantized values $x$ using scale $S$ and zero-point $Z$:$$x = \text{round}\left(\frac{q}{S}\right) + Z$$Execution Runtime & Acceleration: The quantized ONNX graph was converted and executed using Arm NN and CMSIS-NN primitives, unlocking direct hardware acceleration via Arm NEON SIMD vector instructions on AArch64 cores.
Challenges we ran into
Quantization Accuracy Trade-offs: Initial post-training quantization ($\text{FP32} \rightarrow \text{INT8}$) resulted in confidence score drops on ambiguous target objects. Implementing Quantization-Aware Training (QAT) simulated quantization error during the backward pass, restoring model accuracy to within $1.2\%$ of the baseline.Thermal Throttling & Memory Footprint: Sustained 30+ FPS video processing on passive edge devices (like Raspberry Pi) caused CPU core temperatures to spike, leading to thermal throttling. Re-architecting memory buffers to prevent redundant tensor copies stabilized thermal performance under continuous load.Pipeline Thread Contention: Frame ingestion and tensor execution on a single thread created bottlenecking. Moving to an asynchronous producer-consumer model decoupled frame decoding from model execution, ensuring constant FPS without frame dropping.
Accomplishments that we're proud of
100% On-Device Autonomy: Built a fully localized edge vision platform that achieves zero dependency on cloud infrastructure, eliminating API costs, external latency, and connectivity constraints for remote deployments.4x Inference Speedup via INT8 Quantization: Optimized model execution by converting floating-point weights ($\text{FP32}$) to signed 8-bit integers ($\text{INT8}$), achieving a $\sim 4\times$ throughput increase while retaining over $98.8\%$ detection precision through Quantization-Aware Training (QAT).Hardware-Level Vector Acceleration: Unlocked hardware vectorization by compiling tensor operations through Arm NN and CMSIS-NN to directly leverage Arm NEON SIMD instruction sets on AArch64 targets.Sustainable 30+ FPS Edge Performance: Sustained continuous $30+\text{ FPS}$ visual inference on low-cost, passively cooled single-board hardware (such as Raspberry Pi 4/5) without triggering thermal throttling or core memory leakage.Zero-Copy Asynchronous Architecture: Engineered a multi-threaded execution pipeline that decouples frame acquisition from deep learning inference, ensuring zero frame drops during prolonged video processing workloads.
What we learned
Hardware-Aware Design: Tailoring neural network architectures directly to the target CPU's SIMD and vector registers yields massive performance gains without requiring expensive dedicated accelerator hardware.
The Strength of the Arm Ecosystem: Utilizing native tools like Arm NN and CMSIS-NN drastically simplifies low-level hardware acceleration, making high-throughput edge AI accessible across the Arm portfolio.
Optimization Over Scale: Striking the right balance between input resolution, quantization precision, and thread pool size is far more effective for real-time edge processing than simply scaling up model parameters.
What's next for CortexVision
Integration of Arm KleidiAI Micro-kernels: Incorporate Arm's open-source KleidiAI and KleidiCV libraries into the runtime engine to optimize matrix multiplication and convolution routines, exploiting SVE2 and SME2 instruction sets on newer Armv9 CPU architectures. On-Device Hybrid Multimodal AI: Pair the vision pipeline with quantized Small Language Models (SLMs)—such as Phi-3-Mini or Llama-3-1B—compiled via ExecuTorch on Cortex-A cores. This will enable CortexVision to generate continuous, natural-language scene commentary and automated incident summaries entirely offline.Microcontroller Expansion via CMSIS-NN: Port scaled-down micro-models to Arm Cortex-M targets and Ethos NPUs using CMSIS-NN and TensorFlow Lite Micro, creating ultra-low-power, battery-operated smart sensor nodes for zero-maintenance deployments.Federated Edge Governance & Observability: Integrate CortexVision nodes into a unified, privacy-first control plane to monitor node health, aggregate local telemetry, and securely distribute over-the-air (OTA) model weight updates across edge clusters.Field Pilots & Open-Source Community Release: Launch real-world pilot deployments in smart agriculture and localized waste-sorting facilities while open-sourcing AArch64 deployment scripts, Docker containers, and benchmarking tools for the Arm developer community.
Log in or sign up for Devpost to join the conversation.