Wayfinder

Inspiration

Blind and visually impaired travelers face a deeply frustrating reality: modern airline websites are built with dense grids, nested calendars, pop-up promotional banners, and dynamically changing price charts. Traditional screen readers loop endlessly through these layouts, triggering session timeouts before a booking can complete. The result is that 1.3 billion people with disabilities are forced to either ask sighted helpers — sharing passport and credit card details — or give up on independent travel entirely.

We built Wayfinder to flatten that complexity into a natural conversation.

What it does

Wayfinder is a voice-first mobile app that lets blind and low-vision users book flights entirely through speech. A user speaks naturally — "Book a flight from London to Tokyo on June 25th" — and the system searches live airline inventory, presents the best options as narrated audio summaries, and confirms the booking through a tactile hold-to-confirm gesture. A built-in 85% confidence safety net catches any misunderstood utterances and routes them to a human agent with full context.

How we built it

*Voice Architecture: DeepSeek V4 Flash via OpenRouter serves as the LLM orchestrator, extracting structured flight parameters from open-ended natural speech. OpenAI Whisper handles speech-to-text, and OpenAI TTS reads responses aloud.

*Flight Engine: Duffel Sandbox provides global flight inventory and booking APIs. The LLM parses user intent, searches live inventory, and presents options as narrated summaries — no visual browsing required.

*Safety-First Design: Two-layer protection. Every utterance is scored 0–100% for parsing confidence. Below 85%, the system escalates to a human agent with a ticket ID — no booking is ever attempted on uncertain data. A tactile hold-to-confirm button (1.5-second press) prevents accidental voice-trigger bookings.

*Human Handoff: A full CS agent dashboard activates on escalation. Agents can chat, call, or book flights on behalf of the user with plain city name input (e.g., "Tokyo" auto-resolves to NRT).

*Backend: Python FastAPI with SQLite. Frontend: React + Vite + TypeScript with Motion animations. Coded with: Hermes Agent (autonomous AI coding assistant).

Challenges we ran into

*LLM hallucinating foreign languages: The AI parser randomly returned Turkish responses. We added a language sanitizer that detects non-English output and replaces it with hardcoded English fallbacks. (4/6) *Booking TTS not playing: The audio blob was being fetched from the server but never played through the browser. We rewrote the playback pipeline to create Audio elements properly. *Stale closures in React polling: The call polling system stopped after one tick because the closure captured the wrong state. Fixed with ref-based state tracking. *Bookings hidden from trip history: CS agent bookings had no user_id, so the history query returned nothing. Backfilled 11 existing bookings and patched both booking creation paths. *Agent messages silent during calls: The recursive polling function used a stale callState variable. Fixed by using a synchronous ref.

Accomplishments that we're proud of

*End-to-end voice booking works — a blind user can speak a complete itinerary and have it booked in under 5 minutes *85% confidence gate prevents hallucinated bookings — the system knows when it doesn't know and escalates safely *Human handoff is fully functional — CS agents can see context, chat, call, and book flights for users with plain city names *Emergency override — crisis keywords ("missed my connection", "stranded") trigger priority escalation *All bookings sync to My Trips — whether booked by voice or by a CS agent

What we learned

Building for accessibility isn't just about adding alt text — it's about rethinking the entire interaction model. Voice-first design forces clarity: every system response must be concise, every error must have a recovery path, and every action must be reversible. The 85% confidence threshold taught us that knowing when NOT to act is as important as knowing what to do. We also learned that a human-in-the-loop safety net isn't a weakness — it's the feature that makes AI reliable enough to trust with real-world tasks.

What's next for Wayfinder

*Real payment integration — Stripe / Apple Pay with biometric authorization mapped to the hold-to-confirm gesture (5/6) *Native mobile app — iOS/Android with hardware volume-button binding for tactile confirmation *Multi-city and round-trip itineraries — extending the LLM parser beyond A→B single-date bookings *Post-booking concierge — baggage tracking, hotel booking, and flight change notifications *Amadeus GDS integration — full global inventory coverage for production deployment

Built With

• cloudflare-tunnel
• css
• deepseek-v4-flash
• duffel-api
• fastapi
• git
• github
• hermes
• html
• html/css
• html/css-frameworks:-fastapi-(backend)
• javascript
• linux
• motion
• openai-tts
• openai-whisper
• openrouter
• pip
• pip/venv
• pip/venv-tools:-git
• python
• react
• react-+-vite-+-motion-(frontend)-llm:-deepseek-v4-flash-via-openrouter-speech-to-text:-openai-whisper-text-to-speech:-openai-tts-flight-api:-duffel-api-(sandbox)-database:-sqlite-tunneling:-cloudflare-tunnel-infrastructure:-linux
• sqlite
• typescript
• uvicorn
• ven
• vite