ForgetMeNot

A prosthetic for the mind. We use AR and AI to give dementia patients real-time memory injections, instantly identifying loved ones and restoring the human connections stolen by memory loss.

Comment

Inspiration

Watching a loved one with dementia struggle to recognize familiar faces is heartbreaking. We wanted to build something that could act as a gentle, always-present memory companion—like a friend whispering in your ear, "That's Sarah, your granddaughter. Last week you talked about her new job."

What it does

ForgetMeNot is a web-based AR application designed for smart glasses (like Meta Ray-Bans). It:

  • Recognizes faces in real-time using on-device AI
  • Displays contextual info (name, relationship, last conversation summary) as a floating overlay
  • Summarizes conversations using Google Gemini AI so caregivers can log interactions
  • Runs entirely in-browser—no server, no cloud uploads, keeping patient data private

How we built it

  • Frontend: React + Vite + TypeScript for a fast, type-safe UI
  • Face Recognition: face-api.js running client-side (no images leave the device!)
  • Database: IndexedDB via Dexie.js for local, private storage
  • AI Summarization: Google Gemini API to condense conversation notes into friendly reminders
  • AR Simulation: Fullscreen camera view with smooth, physics-based overlays that follow faces

Challenges we ran into

  • Stale Closures in React: The face recognition loop kept "freezing" because React's closures captured old state. We solved this with useRef to always read the latest values.
  • Camera Race Conditions: React's StrictMode caused double-mounting, interrupting the video stream. We implemented an isMounted pattern to prevent stale async operations.
  • Matching Threshold Tuning: Finding the right balance between false positives and missed recognitions required a lot of trial and error.

Accomplishments that we're proud of

  • 100% client-side AI: Face data never leaves the user's device, ensuring privacy for vulnerable patients
  • Real-time performance: Smooth 60fps overlay animations with efficient face detection loops
  • Accessible design: Large fonts, high contrast, warm colors—designed for elderly users

What we learned

  • How to handle async operations safely in React components
  • The nuances of browser-based face recognition and descriptor matching
  • The importance of designing for accessibility from day one

What's next for ForgetMeNot

  • Cloud sync for caregivers to share data across devices
  • Voice assistant integration to speak the person's name aloud
  • Native Meta glasses SDK integration for true AR experience
  • Emotion detection to suggest conversation starters based on the person's mood

Built With

Share this project:

Updates