Inspiration
I spent a year volunteering as a caregiver, and I have grandparents living with memory loss. What stayed with me wasn't the forgetting itself. It was what the forgetting did to people.
Some got hurt: a missed medication, a fall in a room they had walked through a thousand times, a stove left on. But the thing I couldn't stop thinking about was the ones who went quiet. They knew something was wrong with their memory, and they were embarrassed by it. So they stopped asking questions, because every question was proof. They stopped saying their knee hurt, or that they were confused, or that they were scared. They just sat there, and everyone assumed they were fine.
That's the cruel part: the people who need the most help are the least likely to ask for it. Every memory app I looked at assumed the opposite, that the patient would open the app and ask. That is exactly the behaviour dementia takes away.
I wanted to build something that doesn't wait to be asked.
What it does
RememberMe AI listens to the ordinary conversations already happening at home and turns them into memory the patient can get back.
- It remembers the day for them. Conversations are captured and transcribed automatically, then read back each evening as a short spoken recap: who visited, what was said, what happened.
- One conversation, three summaries. A simple second-person recap for the patient, a behavioural summary for the caregiver, and a structured observation log with participant, topics, mood and key concerns.
- Medication reminders that make sense. If the patient mentioned knee pain that morning, the 2pm reminder says so: "This is for pain relief. Earlier today you said your knee was hurting."
- It answers questions from their own day. The patient presses one button and asks "Who came today?" or "Did I take my pills?" The answer comes only from that day's conversations and the people and medications the caregiver entered.
- It tells them who someone is. A photo of a visitor is matched against people the caregiver enrolled, showing their name and relationship.
- A caregiver dashboard. A calendar with a daily mood indicator, a timeline of every conversation with all three summaries, and a chatbot that answers questions about the patient's recent history.
How I built it
The prototype is a multipage Streamlit app with MongoDB Atlas as the care record and Pydantic models for validation. LiveKit handles real-time audio: a small Flask server issues room tokens, and a Python client joins the room, captures each segment and sends it to Whisper for transcription. GPT-4 turns each transcript into the three summaries, and GPT-3.5 Turbo writes the daily recap, the reminders and the caregiver chatbot answers. OpenAI TTS speaks the recaps and reminders aloud. Face recognition uses the face_recognition library on dlib, with encodings stored per person in the database. A background scheduler checks medication times and the recap time twice a minute.
Challenges I ran into
Hallucination is not a cosmetic bug here. An assistant that invents a visitor, a symptom or a dose isn't just wrong, it's dangerous for someone who has no way to check it against their own memory. Every prompt that writes a summary, recap or reminder runs under hard rules: use only facts stated in the transcript, never mention a person who isn't named, never force a connection that isn't there, and write "Unknown" when information is missing. Making that reliable took more iteration than the features themselves.
The patient's speech can't be fully trusted. Confabulation and repetition are symptoms. If the system heard "the doctor said take two more" and changed a reminder, it would create a dosing error with a machine's confidence behind it. So the system reads medications but never edits them. Only the caregiver can change the care plan.
Emergency detection is harder than it looks. My first version matched keywords like "help", which also fires on "can you help me find my glasses". Getting this right without missing a real emergency is still open work.
Accomplishments that I'm proud of
It works end to end on ordinary hardware with no special devices. A conversation goes in, and a spoken recap, a caregiver summary and an observation log come out.
More than that, I'm proud that the design answers the problem I actually saw. A passive system instead of a reactive one wasn't a technical preference. It was the only honest response to watching someone stop asking.
What I learned
The hard part of building for healthcare isn't the model. It's the constraints around it: consent, hallucination, what the system may do on its own, and what it must never do without a human. Almost every good decision I made on this project was a decision about restraint.
I also learned how much design changes when your user can't be relied on to start anything. It rules out most of the patterns you'd normally reach for.
What's finished and what isn't
Working now: conversation capture and transcription, the three summaries, the spoken daily recap, context-grounded reminders, patient Q&A, visitor identification, and the caregiver dashboard and chatbot.
Not yet: emergency phrases show an on-screen message but don't notify the caregiver yet. Speakers are identified by their LiveKit name, not by voice. Prompts are English only. All data in the demo is synthetic, and the system hasn't been tested with real patients or caregivers.
What's next for RememberMe AI
- Caregiver notifications for emergencies, first on the dashboard, then by SMS or push
- Participation tracking over time, so a drop in how much someone speaks shows up as a trend instead of a feeling
- A consent flow for the patient, their representative and regular visitors, with raw audio deleted after transcription
- Arabic and Hebrew support, since most people living with dementia are in low- and middle-income countries
- A FastAPI and React rebuild with a voice-first patient screen
- Testing with real caregivers, which is the only way to find out what I've got wrong
Log in or sign up for Devpost to join the conversation.