RememberMe AI

Dementia patients stop asking for help. RememberMe listens anyway, recalls their day, and tells their caregiver what they didn't say.

Comment

Inspiration

I spent a year volunteering as a caregiver, and I have grandparents living with memory loss. What stayed with me wasn't the forgetting itself. It was what the forgetting did to people.

Some got hurt a missed medication, a fall in a room they'd walked through a thousand times, a stove left on. But the thing I couldn't stop thinking about was the ones who went quiet. They knew something was wrong with their memory, and they were embarrassed by it. So they stopped asking questions, because every question was proof. They stopped saying their knee hurt, or that they were confused, or that they were scared. They just sat there, and everyone assumed they were fine.

That's the cruel part: the people who need the most help are the least likely to ask for it. And every memory app I looked at assumed the opposite that the patient would open the app and ask. That's the exact behaviour dementia takes away.

I wanted to build something that doesn't wait to be asked.

What it does

RememberMe AI listens to the ordinary conversations already happening in the home and turns them into memory the patient can get back.

  • It remembers the day for them. Conversations are captured and transcribed automatically, then summarised into a short spoken recap: who visited, what was said, what happened.
  • It answers questions in the patient's own words. "Who came today?" "Did I take my pills?" The assistant answers from that person's actual recorded day, in simple, warm language, never inventing anything.
  • It writes three summaries from one conversation. A simple second-person recap for the patient, a behavioural summary for the caregiver, and a structured clinical record capturing mood, cognitive state, and concerns.
  • It gives medication reminders that make sense. If the patient mentioned knee pain that morning, the 2pm reminder says so: "This is for pain relief earlier today you said your knee was hurting."
  • It tells them who someone is. Face recognition against caregiver-enrolled profiles answers "who am I talking to?"
  • It watches for silence. Participation is tracked over time, so withdrawal becomes a signal instead of an absence.
  • It escalates emergencies. Distress keywords in patient speech trigger an immediate caregiver alert.

The caregiver gets a dashboard with the full timeline and a chatbot they can ask about the patient's day or week.

How we built it

The working prototype is Python and Streamlit with MongoDB behind it. LiveKit handles real-time audio capture and streaming. Transcription feeds into an LLM summarisation layer, and text-to-speech delivers the recaps and reminders out loud. Face recognition runs on encodings stored per-person in the database. A background scheduler drives the daily recap and the medication reminders.

The pipeline:

Ambient conversation
        ↓
   Transcription
        ↓
Tri-view summarisation  →  patient / caregiver / clinical
        ↓
   Care record (MongoDB)
        ↓
Recap · Reminder · Alert · Visitor ID

Nothing in that chain requires the patient to initiate it.

I'm now rebuilding it as a proper product: a FastAPI backend with a React and TypeScript caregiver dashboard, and a voice-first patient view with large touch targets.

Challenges we ran into

Hallucination is not a cosmetic bug here. An assistant that invents a visitor, a symptom, or a dose isn't just wrong — it's dangerous for someone who has no way to check it against their own memory. Every summarisation prompt in the system is written under hard constraints: use only facts explicitly present in the transcript, never mention a person who isn't named, never force a connection that isn't there. Getting that reliable took far more iteration than the features themselves.

Trusting the patient's speech is more complicated than it sounds. Confabulation and repetition are symptoms. An agent that hears "the doctor said take two more" and silently changes a medication reminder has created a dosing error with a machine's confidence behind it. That pushed me to a tiered design: routine actions like logging a memory happen automatically, but anything clinical becomes a proposal the caregiver confirms.

The rewrite broke things the prototype did fine. Splitting a working monolith into a FastAPI backend and a React frontend surfaced bugs that didn't exist before, especially around real-time audio and background scheduling. The Streamlit version is still the one that runs end to end.

Accomplishments that we're proud of

It works — end to end, on ordinary hardware, with no special devices. A conversation goes in and a spoken recap, a caregiver summary, and a clinical record come out.

More than that, I'm proud that the design answers the problem I actually saw. Building a passive system rather than a reactive one wasn't a technical preference. It was the only honest response to watching someone stop asking.

What we learned

That the hard part of building for healthcare isn't the model. It's the constraints around it — consent, hallucination, what the system is allowed to do on its own, and what it must never do without a human. Every good decision I made on this project was a decision about restraint.

I also learned how much design changes when your user can't be relied on to initiate anything. It rules out most of the patterns you'd reach for by default.

What's next for RememberMe AI

  • Finishing the FastAPI and React build so the product version matches the prototype
  • The agent layer with tiered autonomy, and a pending-actions queue in the caregiver dashboard
  • Longitudinal participation tracking, so a decline in how much someone speaks surfaces as a trend rather than a feeling
  • Multilingual support — the pipeline is language-agnostic, and over 60% of people living with dementia live in low- and middle-income countries
  • Testing with real caregivers, which is the only way to find out what I've got wrong

Source code: github.com/mlkmas/rememberMe-Hack2Heal-2.0

References: WHO Dementia fact sheet · Livingston G, et al. Dementia prevention, intervention, and care: 2024 report of the Lancet standing Commission. The Lancet 2024; 404(10452): 572–628.

Built With

Share this project:

Updates

Submission history