Inspiration

For someone who is blind or has low vision, a simple daily question — "How do I look?" — often means depending on another person. We wanted to give that independence back. Sight is a smart mirror that describes your appearance out loud, so getting ready in the morning doesn't require anyone else.

What it does

Sight is a voice-controlled two-way mirror. Stand in front of it and ask, and it looks at you through a camera and speaks back a clear, natural description of your outfit — the colors, whether they work together, and anything worth flagging like a stain or a mismatched collar. It works two ways: through the mirror's own voice, or through an integrated Amazon Alexa skill so you can ask "Alexa, ask Sight how I look" and hear the answer in Alexa's natural voice. It also shows an ambient dashboard — time, date, a personalized greeting, and live weather — as bright text that appears to float on the glass.

How we built it

• Frontend: React (Vite), rendered fullscreen behind two-way mirror film on a monitor. True-black background keeps the glass reflective while bright widgets show through.
• Backend: Python / Flask, serving the vision and voice endpoints.
• Vision: Google Gemini 2.5 Flash analyzes a live camera frame and generates the spoken outfit description.
• Voice in: the browser Web Speech API for local commands, plus a custom Amazon Alexa Skill.
• Face recognition: face-api.js (TensorFlow.js) in the browser for personalized greetings.
• Accessibility-first: every meaningful output is spoken, not just shown.

Challenges we ran into

• The Alexa camera bridge. Alexa runs in the cloud, but the camera lives in the browser — and Alexa expects a response in under 8 seconds. We solved it by having the browser continuously push its latest frame to the backend, so when Alexa asks, a fresh frame is already waiting to be analyzed and spoken back.
• Gemini output getting cut off. Responses were truncating mid-sentence. We traced it to Gemini 2.5 Flash spending output tokens on internal reasoning; disabling the thinking budget fixed it and made responses faster.
• Hardware. Applying mirror film cleanly, eliminating light leaks, and getting the screen flush against the glass so the widgets don't ghost.

What we learned

How to bridge a cloud voice assistant with local hardware, how to prompt a vision model for genuinely useful spoken descriptions, and how much careful, accessibility-first design shapes every decision.

What's next for Sight

Expanding beyond outfits: reading text aloud (labels, letters, medicine bottles), identifying objects and colors on command, and multi-user profiles — turning Sight into a mirror that describes the whole visual world for blind and low-vision users.

A note on tools

We used AI assistants during development for debugging and code scaffolding, alongside the AI APIs (Google Gemini) that power the product itself.

Built With

• alexa-skills-kit
• face-api.js
• flask
• gemini
• google-gemini-api
• html
• javascript
• ngrok
• python
• react
• tensorflow.js
• vite
• web-speech-api