SightLine

• 

  homescreen

• 

  Working

• 

  Asking

• 

  Working

Inspiration

People with low vision who find it difficult to know what's happening around them without someone else there to describe it. Dedicated assistive hardware for this costs $300–600, which is out of reach for a lot of people. I wanted to see if a phone camera and a good AI model could do the same job for free. This fits the Software Development track — a full working application solving a real accessibility gap, built end-to-end with React, a Vercel serverless backend, and the Gemini API.

How we built it

The frontend is React and Vite, styled with Tailwind. The camera feed is captured with getUserMedia, and a snapshot is sent to a Vercel serverless function, which calls the Gemini API for scene description. The response is read aloud automatically using the Web Speech API's SpeechSynthesis, and follow-up questions can be asked by voice using SpeechRecognition. The whole thing is deployed on Vercel with GitHub auto-deploy.

Challenges we ran into

Getting the API key handling right without exposing it to the browser, keeping the free-tier rate limits from interrupting a live demo (solved by rotating across multiple API keys automatically if one hits its limit), and making sure the interface itself was genuinely accessible — full keyboard focus states and aria-live captions — so it didn't create a second accessibility problem while solving the first one.

Accomplishments that we're proud of

A fully working end-to-end voice loop — point, tap, listen, ask a follow-up — running entirely in the browser with no app install. And building it solo within a tight timeframe while keeping the interface genuinely accessible, not just functional.

What we learned

How much thought accessible design actually requires beyond just "the feature works" — safety-relevant details need to come first in a spoken description, not last. Also a lot about handling API rate limits gracefully so a demo doesn't visibly break under real usage.

What's next for SightLine

Ideas not yet built: offline/low-bandwidth mode for a compressed description when signal is poor, a saved-session history so descriptions can be revisited, and multi-language output.

Built With

• gemini-api
• javascript
• react
• serverless-functions
• tailwind-css
• vercel
• vite
• web-speech-api