https://drive.google.com/file/d/1N0Dy-Ih1SHMCcf062uZpm7JyizWqmKJ-/view?usp=sharing LINK FOR THE VIDEO ^
Inspiration
I built it for my grandmother, but it's for anyone navigating the world with vision loss.
What it does
Second Sight is a hands-free vision companion for people with impaired vision, powered by Gemini's multimodal API. Point a phone camera at something and ask a question out loud — "what pill is this?", "what bill is this?" — and Gemini reads and answers in real time. It also runs Hazard Monitoring silently in the background, only speaking up if it detects a real hazard like a stove left on or an unsafe street crossing. If the user says "help" or "I feel dizzy," it triggers a simulated emergency call that speaks their saved medical profile (name, age, blood type, conditions, nearest hospital) aloud. It also handles pill reminders and remembers conversation context, so a follow-up like "is it expired?" actually makes sense
How we built it
I used gemini and claude to code a project in html, then depolyed it using netlify.
Challenges we ran into
Wouldnt be able to properly describe what it was seeing and had to go through different renditions of the model.
Accomplishments that we're proud of
Being able to deploy a working demo project
What we learned
That with time constraints not all the ideal features are able to be implemented.
What's next for SecondSight
Build an acutal lanyard hardware where it can run off with a camer and mic.
Log in or sign up for Devpost to join the conversation.