-
starting simple page!
-
-
-
-
-
Inspiration
We wanted to build something that truly empowers blind and low-vision people in their daily lives. Too many accessibility tools are complicated, require multiple steps, or feel like an afterthought. We imagined a single, simple interface where you just hold the screen and speak—no complex menus, no visual scanning. Faith was born from the belief that powerful AI should be accessible through the most natural interface: your voice.
What it does
Faith is a voice-first iPhone app designed specifically for blind and low-vision users. The entire screen is one big hold-to-talk area:
Simple interaction: Hold anywhere on the screen, speak your request, and release. That's it.
Smart routing: Say an object like "where are my keys" and Faith launches the object finder. Say a travel request like "book a train to Boston Friday" and it connects you to the booking agent.
Object Finder with Spatial Audio: Using ARKit and the camera, Faith helps you locate lost objects. Before any photo leaves your phone, Faith asks for explicit consent: "Send this view to find it? Hold the screen to send, or double tap to cancel." Once you confirm, the photo goes through Faith's backend to Baseten's Kimi-K3 vision model, which identifies the object's location. Faith then guides you with 3D spatial audio and head tracking—turn your head and the audio moves, guiding you directly to the item. If the model can't place the object with confidence, Faith tells you honestly rather than guessing.
Voice Booking Agent: An ElevenLabs Conversational AI agent with a calm, soothing voice (Sarah, slower speed for accessibility). It searches travel options and reads them one at a time. Hold the screen 1.5 seconds to accept, double-tap to reject and hear the next option, or just say yes/no. It never confirms a booking without a clear "yes." All bookings are demo tickets (clearly announced), with no real payments. The agent can also check venue accessibility information and hand off to the object finder mid-conversation.
Privacy-First Memory: With your permission, Faith remembers preferences like your home city, accessibility needs, and usual travel party size—stored securely in SurrealDB Cloud. Say "forget my preferences" anytime to delete everything.
Real-Time Information: Using Anchor Browser, Faith runs a cloud browser to read real schedules and prices from providers like Amtrak, Greyhound, and AMC. Results are labeled as unverified and read-only; actual booking stays demo-only.
How we built it
iOS Frontend: Built with Swift and SwiftUI, using ARKit for object detection, AVFoundation for spatial audio, CoreMotion for head tracking, and the Speech framework for voice input. The ElevenLabs Swift SDK (LiveKit WebRTC) powers real-time voice conversations.
Backend: Python FastAPI hosted on Daytona (public sandbox URL so no secrets live in the app). We use SQLite for transactional data, httpx for HTTP requests, and Pydantic for validation. The backend has over 70 tests ensuring reliability.
AI & Vision: Baseten's Kimi-K3 vision model returns normalized bounding boxes for object locations (we disabled reasoning mode for instant responses). AWS Bedrock serves as an optional fallback vision model.
Voice & Audio: ElevenLabs powers both the conversational AI agent and all prerecorded soothing voice prompts throughout the app.
Data & Observability: SurrealDB Cloud stores user preferences and mirrors bookings. Arize provides privacy-safe tracing and observability—no raw text, audio, or identity is sent to traces, only anonymized metrics.
Real-World Data: Anchor Browser fetches live ticket times, prices, and venue accessibility info by running headless browser sessions in the cloud.
Testing: Over 200 tests on iOS, 70+ on the backend. We use XcodeGen for project management.
Challenges we ran into
Vision model outputs: Getting a vision model to return normalized bounding boxes instead of pixel coordinates was tricky. We switched to Kimi-K3 and disabled its reasoning mode so it answers instantly without lengthy explanations.
Blind-safe gestures: Designing interactions that are safe for blind users took careful thought. We landed on: hold to accept (requires deliberate action), double-tap to reject (familiar iOS gesture), and no accidental confirms. Every action has voice feedback.
SurrealDB auth: Setting up database-level authentication with proper roles took debugging, but we got it working with the EDITOR role for the backend user.
Daytona deployment: Hosting on Daytona with dependency conflicts was challenging. We solved it by using a Python virtual environment to isolate packages.
Privacy consent: Ensuring per-frame camera consent without disrupting the experience required careful UX design. Users explicitly approve each camera view before it's sent.
Accomplishments that we're proud of
True voice-first design: The entire app is one tap target. No complex navigation for blind users.
Spatial audio guidance: Hearing the location of an object in 3D space and being guided to it with head tracking feels magical.
Honest AI: Faith never pretends to know something it doesn't. If it can't find an object with confidence, it says so.
Privacy by design: Per-frame camera consent, short-lived tokens, no secrets in the client, rate limits, and privacy-safe observability.
Accessibility in booking: The agent checks venue accessibility info and never rushes you—hold 1.5 seconds to accept, ensuring no accidental bookings.
Comprehensive testing: Over 270 tests combined give us confidence the app works reliably.
What we learned
Accessibility is about trust: Blind users need to trust that the app won't trick them or make decisions without permission. Every interaction should feel safe and predictable.
Voice UI is harder than it looks: Timing, feedback, and clear confirmation patterns are crucial. We learned to slow down, give explicit feedback, and never assume.
ARKit is powerful: Combined with spatial audio and head tracking, ARKit can create experiences that genuinely help people navigate the world.
Cloud services matter: Using Daytona (backend hosting), ElevenLabs (voice AI), Baseten (vision), SurrealDB (data), Anchor (real-world info), and Arize (observability) let us build something ambitious quickly.
Test everything: With accessibility apps, bugs aren't just annoying—they can be disorienting or even unsafe. Heavy testing is essential.
What's next for Faith
Real payments: Partner with a trusted payment provider to enable actual bookings while maintaining our safety-first approach.
More providers: Expand beyond trains, buses, and movies to flights, hotels, restaurants, and events.
Offline object finding: Use on-device ML models so you can find objects even without internet.
Apple Watch integration: Haptic feedback on your wrist to guide you to objects, keeping your hands free.
Multi-language support: Make Faith accessible to non-English speakers.
Community features: Let users share accessible venue tips and reviews.
Log in or sign up for Devpost to join the conversation.