-
project image 1
-
project image 2
Inspiration
The genesis of The Third Eye stemmed from observing a stark technological divide in assistive devices. Today, artificial intelligence can drive cars and generate art, yet the primary navigational tool for the visually impaired remains the rudimentary white cane. While high-end cognitive wearables do exist, their exorbitant costs—often exceeding ₹2,00,000—effectively act as a prohibitive "disability tax." We were inspired by an urgent need to democratize machine perception. The goal was to engineer a system that delivers the cognitive power of an elite AI wearable at a fraction of the cost, ensuring that independent mobility and digital literacy are accessible rights, not exclusive luxuries.
What it does
The Third Eye functions as a wearable "Digital Visual Cortex" that provides simultaneous physical safety and semantic awareness. It replaces the passive white cane with an active, intelligent companion using a dual-engine system:
The Reflex Engine: Operates entirely offline to detect dynamic physical hazards (like approaching vehicles or structural barriers) and delivers directional audio alerts in under 50 milliseconds.
The Cognitive Engine: An on-demand, voice-activated intelligence layer. By saying the wake word ("Edith"), the glasses analyze the scene using GPT-4o, allowing the user to read printed text, identify currency, and understand complex social environments through natural speech.
How we built it
The system was engineered around a proprietary Hybrid Split-Core Architecture to balance extreme low latency with deep cognitive intelligence on frugal hardware. We used a Raspberry Pi Zero 2 W as the primary computational hub, executing a quantized TensorFlow Lite model for offline edge-detection, while routing asynchronous API calls to OpenAI for cognitive tasks. To prevent the main CPU from bottlenecking, we interfaced an Arduino Nano to handle low-latency hardware interrupts and basic sensor polling. The camera module continuously captures the user's field of view, while audio feedback is discreetly relayed through bone-conduction transducers, ensuring the user's natural hearing is never obstructed.
Challenges we ran into
Developing a simultaneous dual-AI system on severely constrained hardware presented significant engineering hurdles:
Compute & Thermal Throttling: Pushing real-time video inference and audio resampling on a microcontroller with only 512MB of RAM pushed the Pi near its thermal limits. We solved this by engineering an intelligent frame-throttling algorithm and aggressively quantizing our neural networks to run within a safe thermal envelope.
Asynchronous Blocking: Ensuring the cloud-based API calls didn't crash the real-time local hazard detection loop required complex multi-threading. We had to master Python's asyncio and Mutex locks to safely share the camera frame between the edge and cloud pathways.
Power Instability: Simultaneous vision inference and audio transmission caused voltage drops. We mitigated this by designing a decoupled power architecture using a dedicated 24000 mAh power bank to isolate the microcontrollers.
Accomplishments that we're proud of
Zero-Latency Offline Safety: We successfully achieved a sub-50ms processing latency for hazard detection that requires absolutely no internet connection, guaranteeing the user's physical safety in any environment.
Frugal Innovation: We completely re-engineered the visual computing stack using Commercial Off-The-Shelf (COTS) components, bringing the total hardware cost down to under ₹8,000—a 96% cost reduction compared to industry alternatives.
Seamless Integration: Successfully bridging hardware-level edge computing (TFLite) with generative cloud AI (GPT-4o) into a single, cohesive user experience.
What we learned
This project was a masterclass in frugal engineering and embedded systems. We gained profound, hands-on insights into model quantization—specifically the intricacies of shrinking a neural network to run on minimal hardware without sacrificing critical detection accuracy. We also learned how to strategically orchestrate multi-threaded operations and manage shared memory queues, proving that highly complex AI architectures can be successfully deployed on highly accessible, low-power hardware.
What's next for The Third Eye
The current prototype proves the viability of the architecture, but we aim to evolve it into a fully integrated visual prosthesis. Our next steps include:
Infrared & ToF Sensing: Integrating Time-of-Flight sensors to ensure the reflex engine can operate flawlessly in complete darkness.
Hardware Miniaturization: Consolidating the Pi, Arduino, and power management circuits onto a single, custom-designed Printed Circuit Board (PCB) to embed the technology seamlessly into standard eyeglass frames.
Dedicated Edge Accelerators: Incorporating micro-NPUs (like the Google Coral Edge TPU) to run Small Language Models (SLMs) entirely locally, further reducing our reliance on cloud APIs.
Log in or sign up for Devpost to join the conversation.