AuraPath — Real-Time Assistive Hazard Alert System

💡 Inspiration

Over 250 million visually impaired individuals navigate daily physical environments full of dynamic spatial hazards—from approaching pedestrians to parked vehicles and street obstacles. Cloud-based AI solutions suffer from network latency, high cellular costs, and severe privacy concerns.

We built AuraPath to prove that a 100% offline, real-time, low-power walking-hazard alert system can run entirely on a budget Raspberry Pi 4 (Armv8 Cortex-A72) equipped with a camera module and a wearable speaker.

🚀 What It Does

AuraPath continuously perceives the user's forward walking path, detects physical hazards, and outputs low-latency, non-blocking verbal cues ("Person ahead close", "Vehicle left approaching"):

1. Perception Engine: Processes live camera frames at high framerates using an Arm-optimized neural network.
2. Spatial Hazard Evaluator: Computes relative bounding box area into proximity zones (CRITICAL, CLOSE, MEDIUM, FAR) and maps center X-coordinates to directional sectors (LEFT, AHEAD, RIGHT).
3. Priority Queue & Cooldown Engine: Ranks hazards by threat severity (Vehicle > Person > Obstacle) and suppresses repetitive alert spam using hysteresis timing.
4. Asynchronous Audio Alert System: Synthesizes non-blocking offline TTS cues without interrupting frame acquisition or inference loops.

🛠️ How We Built It & Arm Optimization Strategy

We designed AuraPath from the ground up for maximum edge efficiency on Arm architecture:

• Baseline Model: Exported PyTorch YOLOv8n to FP32 ONNX format (640x640 input resolution).
• Arm INT8 Quantization: Applied Dynamic Post-Training Quantization (PTQ) to convert 32-bit floating-point weights into 8-bit unsigned integers (QUInt8).
• Arm XNNPACK & NEON Acceleration: Deployed the model on ONNX Runtime using the XNNPACKExecutionProvider, leveraging 128-bit SIMD vector instructions on the Cortex-A72 CPU for high-frequency integer matrix multiplication.
• Threaded Non-Blocking Architecture: Implemented a decoupled 3-thread pipeline (Frame Grabber, Inference/Logic Engine, TTS Queue Manager) so audio speech synthesis never freezes frame capture.

📊 Measured Arm Optimization Proof (Before vs. After)

🧠 What We Learned & Challenges Overcome

• Preventing Audio Stalls: Spoken audio synthesis initially blocked the main loop. We resolved this by building an asynchronous thread queue worker.
• Monocular Proximity Heuristics: Without expensive depth sensors, using bounding box relative area and center-x geometry allowed us to achieve reliable spatial awareness.
• Arm Vector SIMD Gains: We gained firsthand experience seeing how INT8 quantization combined with Arm NEON/XNNPACK execution providers can transform low-cost microprocessors into life-saving assistive devices.

Built With

• arm
• espeak-ng
• int8-quantization
• onnx-runtime
• opencv
• physical-ai
• python
• pyttsx3
• raspberry-pi
• xnnpack
• yolov8