SightLine

Interface
Asking/Input
working
Working

Inspiration

People with low vision who find it difficult to know what's happening around them without someone else there to describe it. Dedicated assistive hardware for this costs $300–600, which is out of reach for a lot of people. I wanted to see if a phone camera and a good AI model could do the same job for free. This fits the Software Development track — a full working application solving a real accessibility gap, built end-to-end with React, a Vercel serverless backend, and the Gemini API.

How we built it

The frontend is React and Vite, styled with Tailwind. The camera feed is captured with getUserMedia, and a snapshot is sent to a Vercel serverless function, which calls the Gemini API for scene description. The response is read aloud automatically using the Web Speech API's SpeechSynthesis, and follow-up questions can be asked by voice using SpeechRecognition. The whole thing is deployed on Vercel with GitHub auto-deploy.

Challenges we ran into

Getting the API key handling right without exposing it to the browser, keeping the free-tier rate limits from interrupting a live demo (solved by rotating across multiple API keys automatically if one hits its limit), and making sure the interface itself was genuinely accessible — full keyboard focus states and aria-live captions — so it didn't create a second accessibility problem while solving the first one.

Accomplishments that we're proud of

A fully working end-to-end voice loop — point, tap, listen, ask a follow-up — running entirely in the browser with no app install. And building it solo within a tight timeframe while keeping the interface genuinely accessible, not just functional.

What we learned

How much thought accessible design actually requires beyond just "the feature works" — safety-relevant details need to come first in a spoken description, not last. Also a lot about handling API rate limits gracefully so a demo doesn't visibly break under real usage.

What's next for SightLine

Ideas not yet built: offline/low-bandwidth mode for a compressed description when signal is poor, a saved-session history so descriptions can be revisited, and multi-language output.