-
déjà-view helps people with memory loss and facial blindness recognize their loved ones and pick up right where they left off.
Inspiration
One friday night, our best friend got a video call from his grandfather. His twin brother grabbed the phone to say ,how are you doing dada" , and his grandfather looked at him blankly and asked, "Who are you?"
He hadn’t forgotten his grandson. He has face blindness (prosopagnosia), which makes it impossible to recognize even the faces of people you love. Watching a grandson who adores him get met with a stranger’s stare broke our hearts.
About 2.5% of the world, roughly 412 million people, live with face blindness. They face this every day, and it costs them the closeness they deserve with family and friends. We wanted to build something so that moment never has to happen again.
What it does
Deja View simulates what a person with face blindness would experience wearing smart glasses, or an affordable camera rig for anyone without them. The camera recognizes faces the user has registered and shows three things about the faces: the person’s name, their relationship to the user (mother, brother, father, sister etc), and key points from their last conversation. That’s enough to connect the face to the bond, so users can greet family and friends with confidence. This changes the game by helping the face blind, and you automatically see better bonding around them. On camera only setups, the Gemini API processes what the camera sees, and ElevenLabs text to speech turns it into a natural human voice that tells the user about the person, and also the last conversation key points you had which is crucial for memory recall.
How we built it
We built the interface with React , TypeScript and TailwindCSS. A live facial recognition video feed sits in the center. It displays the details of the recognized person in a comfort way so user with face blindness can follow what’s happening at a glance.
When a face is detected, we convert it into a 512 number embedding using InsightFace 2.1 module. We compare embeddings with NumPy and store them in SQLite database. When a face matches one in the database, the app shows that person’s details right beside their face which was stored in the database
- Registration: When a new person is added, ElevenLabs Scribe transcribes what’s said in the conversation between user and other indvidiual, and the Gemini API extracts the person’s name and their relationship to the user from the sentence.
- Conversation recall: Gemini follows each conversation and returns the top 3 most relevant points front,ranked with a weighted scoring system. These are the points most likely to help the user remember who the person is and what they last talked about.
- Voice: Gemini helps to boil the entire conversation history saved to a few short sentences, and we pass them to the ElevenLabs API to produce a natural human voice that tells the user who they’re looking at. This creates a seamless experience
Challenges we ran into
- Our first version could find faces but couldn’t tell people apart. But soon, we trained our module using more images and fine tuned it enough for it to recognize faces and different angles, shadows etc
- Understanding how to store faces in database was a problem. We researched and computed the faces into vectors and stored it in form of embedding in database
- Visualizing where the information of people recognized shall be displayed in a calm comfort way for the user was a challenge. We found the solution by being minimalistic and staying consistent to the problem statement
Accomplishments that we're proud of
1) Training and implementing a model which accurately recognizes human face and pulls the match from the databse 2) understanding and learning how face recognition works in form of vectors 3) Saving the name, relationship, conversations to help people suffering from face blindness
What we learned
Face recognition is about being careful, not just accurate. Using InsightFace, we learned that every face becomes 512 numbers and recognition is just comparing them. The real lesson was deciding when to say “Unknown.” For someone with dementia, a wrong name is worse than no name, so we built strict rules that only show a name when the app is truly confident.
AI must never invent a memory. When we used Gemini to summarize conversations, we learned that AI can confidently make up details. So every fact déjà-view saves must point to a real quote from the conversation, and nothing is stored until a person reviews it.
Free AI has real limits. Gemini’s free-tier credits and rate limits ran out quickly during testing. We learned to cut unnecessary API calls, handle “too many requests” errors gracefully, and make the AI provider switchable so our demo never depends on a single service.
Voice makes it human. Integrating ElevenLabs for speech-to-text and natural voice taught us that the experience matters as much as the technology. A gentle spoken reminder feels personal in a way text on a screen can’t. We also learned to pause the microphone while the app speaks, so it never transcribes its own voice.
Privacy has to be built in from day one. Faces never leave the device, and no one is registered or recorded without a clear button press. For a tool this personal, we learned that trust is a feature, not an afterthought.
What's next for déjà-view
From phone and laptop to smart glasses Today déjà-view uses a phone camera to see who is in front of the facial blind, and the laptop recognizes the person and shows who they are. Next, we’ll fit everything into one pair of lightweight smart glasses. The camera sits in the frame, and the reminder appears in the corner of the patient’s view or is whispered in their ear the moment a loved one walks in. There’s no device to hold, so the patient can greet people naturally.
From reminding to re-learning déjà-view won’t just tell the patient who someone is; it will help them remember on their own. Each visit, it repeats the same few easy-to-notice details about the person, such as their hairstyle, height or a familiar habit. Over time it gives a hint instead of the answer and lets the patient try to recall the name first. This follows spaced retrieval, a research-backed memory method. As the patient improves, the hints fade, and more visits begin with real recognition.
Recognition beyond the face We’ll add voice recognition, so déjà-view can still recognize a loved one when their face isn’t visible: when they’re turned away, in a dim room, or speaking on the phone.
Log in or sign up for Devpost to join the conversation.