Inspiration
For someone who is blind or has low vision, a simple daily question — "How do I look?" — often means depending on another person. We wanted to give that independence back. Sight is a smart mirror that describes your appearance out loud, so getting ready in the morning doesn't require anyone else.
What it does
Sight is a voice-controlled two-way mirror. Stand in front of it and ask, and it looks at you through a camera and speaks back a clear, natural description of your outfit — the colors, whether they work together, and anything worth flagging like a stain or a mismatched collar. It works two ways: through the mirror's own voice, or through an integrated Amazon Alexa skill so you can ask "Alexa, ask Sight how I look" and hear the answer in Alexa's natural voice. It also shows an ambient dashboard — time, date, a personalized greeting, and live weather — as bright text that appears to float on the glass.
How we built it
- Frontend: React (Vite), rendered fullscreen behind two-way mirror film on a monitor. True-black background keeps the glass reflective while bright widgets show through.
- Backend: Python / Flask, serving the vision and voice endpoints.
- Vision: Google Gemini 2.5 Flash analyzes a live camera frame and generates the spoken outfit description.
- Voice in: the browser Web Speech API for local commands, plus a custom Amazon Alexa Skill.
- Face recognition: face-api.js (TensorFlow.js) in the browser for personalized greetings.
- Accessibility-first: every meaningful output is spoken, not just shown.
Challenges we ran into
- The Alexa camera bridge. Alexa runs in the cloud, but the camera lives in the browser — and Alexa expects a response in under 8 seconds. We solved it by having the browser continuously push its latest frame to the backend, so when Alexa asks, a fresh frame is already waiting to be analyzed and spoken back.
- Gemini output getting cut off. Responses were truncating mid-sentence. We traced it to Gemini 2.5 Flash spending output tokens on internal reasoning; disabling the thinking budget fixed it and made responses faster.
- Hardware. Applying mirror film cleanly, eliminating light leaks, and getting the screen flush against the glass so the widgets don't ghost.
What we learned
How to bridge a cloud voice assistant with local hardware, how to prompt a vision model for genuinely useful spoken descriptions, and how much careful, accessibility-first design shapes every decision.
What's next for Sight
Expanding beyond outfits: reading text aloud (labels, letters, medicine bottles), identifying objects and colors on command, and multi-user profiles — turning Sight into a mirror that describes the whole visual world for blind and low-vision users.
A note on tools
We used AI assistants during development for debugging and code scaffolding, alongside the AI APIs (Google Gemini) that power the product itself.
