MeowMeow - AuraSight

AuraSight is an AI companion for the blind. Worn on a lanyard, it uses a phone camera and Reka AI to scan head-level hazards in real-time, delivering hands-free audio alerts to restore autonomy.

Comment

1. Executive Summary

Urban navigation is a high-cognitive-load task for visually impaired pedestrians. Traditional tools like white canes and GPS handle macro-routing but leave a dangerous blind spot for upper-body and head-level hazards. AuraSight bridges this gap. By leveraging a neck-worn smartphone camera, continuous video chunk streaming, and real-time auditory feedback, it establishes a passive safety net for the user.

2. Problem Definition

Traditional mobility aids create a dangerous physical blind spot. While GPS charts the macro-route, it cannot detect micro-level immediate hazards like low-hanging construction barriers or sudden street works.

  • 70% of blind individuals report experiencing head-level collisions.
  • 10.5% of all pedestrian injuries in this demographic involve head/neck trauma.
  • 30% completely avoid traveling to unfamiliar locations due to navigation anxiety.

3. Proposed Solution — AuraSight

A seamless, four-tier accessibility architecture:

  1. Camera Hazard Scanning: Captures 2.5s video chunks via a neck-lanyard smartphone.
  2. Spoken Warnings: Translates context directly to speech using the Web Speech TTS API.
  3. Hands-Free Voice Controls: Accepts system commands via the Web Speech Recognition API.
  4. Walking Directions: Maps active routes using OpenStreetMap Nominatim and OSRM.

4. Technical Architecture & AI Implementation

Full-stack monorepo deployed on Vercel. The frontend is a lightweight React SPA built with Vite to maximize browser-native API performance. The backend is a Python FastAPI service that extracts frames via ffmpeg and sends them to the Reka AI (reka-flash) multimodal model.

To overcome serverless statelessness, the system utilizes Lazy Session Registration, embedding unique session contexts inside each payload chunk to eliminate the need for persistent database locks during active navigation.

5. Solution Evaluation & Feasibility

High spatial understanding with localized execution boundaries.

  • Zero Infrastructure Overhead: Externalizing frame analysis to Reka AI removes the need for costly private cloud GPU configurations.
  • Predictable Economics: Compute metrics scale linearly with active user engagement.

Future Horizon Roadmap

  1. Phase 1: Transition the web platform into native wrappers (React Native/Flutter) for background camera execution permissions.
  2. Phase 2: Integrate localized Small Language Models (SLMs) via WebNN for offline processing safety during network dropouts.
  3. Phase 3: Migrate inputs to wearable smart glasses and outputs to Bluetooth LE Audio systems.

6. Conclusion

AuraSight delivers crucial spatial awareness without specialized hardware requirements. By combining frontier vision models with native web technologies, the platform introduces an immediate, cost-efficient layer of autonomy and physical protection for visually impaired individuals navigating rapidly changing urban environments.

Built With

Share this project:

Updates