BlindNav

"BlindNav turns Amazon Echoes into a 3D indoor GPS sound trail, guiding 285M visually impaired people safely around home obstacles with zero new hardware."

Comment

Inspiration

Over 285 million people worldwide live with visual impairment. For them, routine tasks like finding heart medication, car keys, or reading glasses can turn into stressful, disorienting ordeals.

When a blind user asks current smart speakers: "Alexa, where is my medication?", the assistant typically responds: "It is on the primary bathroom counter."

While factually accurate, telling someone WHERE an item is does not help them safely GET there. Visually impaired individuals are left to navigate hallways blindly, running the risk of colliding with doorframes or tripping over dropped obstacles like shoes, packages, or toys.

We asked an audacious question: What if we could turn the Amazon Echo devices already distributed across millions of homes into a phased 3D spatial acoustic GPS?

That vision became BlindNav—an intelligent, hands-free indoor navigation co-pilot powered by Alexa+, AWS Bedrock, and open-standard Model Context Protocol (MCP) agents, requiring $0 in new hardware.

What it does

BlindNav transforms ordinary smart speakers into a dynamic indoor acoustic beacon array that guides visually impaired users step-by-step using 3D directional sound:

  1. Acoustic Breadcrumb Trails: When a user asks Alexa to find an item, BlindNav computes a safe corridor through the home's topological room graph. As the user walks, nearby Echo speakers emit resonant, calibrated acoustic beacons (e.g. 440 Hz in the living room, 523 Hz in the hallway, 783 Hz at the destination). The volume and stereo pan dynamically shift to physically "pull" the user along the path.
  2. Proactive Hazard Avoidance via Ring Vision: Ring indoor cameras and floor sensors continuously feed dropped-object alerts into BlindNav's obstacle graph. If running shoes or delivery boxes block a hallway, BlindNav recalculates the trajectory in real-time, sounding an acoustic double-pulse alert and giving precise voice bypass instructions: "Caution: Running shoes detected 3 paces ahead on your right. Step two paces left."
  3. Instant Voice Grounding & Item Indexing: Users can register any item verbally: "Alexa, I'm putting my walking cane next to the umbrella stand." BlindNav's MCP agent extracts the physical landmark and indexes it into the AWS Bedrock spatial graph with timestamp verification.
  4. Hands-Free Autonomy: Operates 100% through ambient voice and spatial sound—no phone screen, no sighted human helper, and no wearable glasses needed.

How we built it

BlindNav is engineered around Amazon's cutting-edge developer stack across the Alexa+ Track and the AWS Builder mini challenge:

  • Self-Hosted Alexa+ MCP Server (Streamable HTTP): Built using Python and the Model Context Protocol specification (v2025-11-25+). It runs as an asynchronous SSE stream exposing tools: find_object, navigate_to_object, get_home_layout, register_object, get_obstacles, and get_spatial_audio_state.
  • AWS Bedrock (Claude 3.5 Sonnet): Acts as the high-level spatial reasoning brain. Bedrock consumes topological room coordinates, doorway dimensions, and active hazards to synthesize concise, empathetic voice directions optimized for voice latency.
  • Topological Graph Pathfinding: Employs an obstacle-cost-weighted Dijkstra algorithm across room adjacency graphs, ensuring walking paths prioritize wide doorways and avoid dynamic clutter.
  • Acoustic Propagation Engine: Models multi-speaker acoustic attenuation using an inverse-distance decay function: $$I(d) = I_0 \cdot \frac{1}{1 + \alpha d^2}$$ where $I_0$ represents base output amplitude, $d$ is the Euclidean distance from the user coordinate to the Echo node, and $\alpha$ is the acoustic room absorption coefficient.
  • Frontend Live Console: Developed in React 18, Vite, and Tailwind CSS using a Jet Black (#0A0A0A) and Cerulean Blue (#007BA7) architectural blueprint theme. Features interactive SVG floor plans with door-swing arcs, CAD furniture silhouettes, expanding radar wavefronts, and Web Audio API spatial acoustic synthesis.

Challenges we ran into

  1. Streamable HTTP (SSE) Reverse-Proxy Buffering: The new Alexa+ MCP spec (Nov 2025+) requires true streaming Server-Sent Events for bi-directional heartbeats. Localhost development proxies initially buffered SSE event chunks. We resolved this by configuring explicit text/event-stream headers, disabling proxy buffering via X-Accel-Buffering: no, and crafting asynchronous event generators in FastAPI.
  2. Multi-Speaker Synchronization & Latency: Cloud-based text-to-speech round-trips (400–800 ms) were too slow for rapid turn-by-turn proximity warnings. We solved this with a hybrid edge-cloud architecture: lightweight resonant sine waveforms are pre-cached and triggered directly across Echo nodes, while cloud Bedrock handles semantic scene reasoning.
  3. Intuitive Non-Visual Spatial Cues: Designing sound that communicates direction without visual confirmation required rigorous tuning of frequency spacing ($\Delta f$) and stereo panning envelopes to prevent acoustic disorientation in enclosed rooms.

Accomplishments that we're proud of

  • $0 Additional Hardware: Unlocks genuine accessibility for 285M people using hardware already in their homes.
  • Full Alexa+ MCP Compliance: Implemented a self-hosted Streamable HTTP server fulfilling the strict hackathon technical track criteria.
  • Integrated Ring Floor Vision: Turned security cameras into proactive accessibility sensors that protect users from tripping hazards.
  • Realistic Architectural Blueprint: Created an interactive CAD floor plan with door swings, radar sweeps, and live Web Audio synthesis that lets judges hear the spatial audio in their own headphones.
  • Documented Friction Logs: Submitted detailed friction logs on MCP streaming and edge caching for the 10% judging bonus.

What we learned

  • The Power of the MCP Standard: Building with the Model Context Protocol proved how transformative open agent standards are for voice ecosystems. Alexa+ can dynamically discover and execute spatial tools with zero custom client glue code.
  • Designing for Acoustic Intuition: In accessibility technology, less is more. Short, melodic harmonic pings coupled with clear directional speech are infinitely more reassuring than continuous alarms.

What's next for BlindNav

  • Ultra-Wideband (UWB) Centimeter Tracking: Leveraging UWB in newer Echo devices to track user movement down to 5 cm accuracy.
  • Outdoor-to-Indoor Handoff with Ring: Expanding guidance from the driveway or porch through the front door using Ring Doorbell chime beacons and path lighting.
  • Open-Source Community Mesh: Publishing the BlindNav MCP Server as a public PyPI/Docker package so developers globally can contribute custom smart home beacon integrations.

Built With

  • accessibility
  • acoustic-beacons
  • alexa+
  • aws-bedrock
  • claude-3.5-sonnet
  • dijkstra-algorithm
  • fastapi
  • iot-core
  • model-context-protocol
  • open
  • python
  • react-18
  • ring-api
  • server-sent-events
  • smart-home
  • source
  • spatial-audio
  • streamable-http
  • tailwind-css
  • vite
  • web-audio-api
Share this project:

Updates

Submission history