Patient dashboard with analytics useful for the caretaker
Speech network graph of a patient's speech patterns (cycles indicate higher severity of dementia)
Conversation session with AI moderator and other AI patients
Quiz portion of the therapy after a conversation session

Patronum: AI-powered Cognitive Stimulation Therapy for Dementia Patients

Inspiration

https://www.reddit.com/r/ArtificialInteligence/comments/1bsf9mi/my_parents_are_getting_dementia_can_ai_help/

Dementia is a progressive neurological condition affecting over 55 million people worldwide. It impairs memory, language, reasoning, and the ability to perform daily activities. While there is currently no cure, treatments focus on slowing cognitive decline, maintaining function, and improving quality of life. One evidence-based approach to combating dementia is Cognitive Stimulation Therapy (CST): a structured program of themed activities designed to actively stimulate thinking, concentration, memory, and social interaction.

CST sessions, typically conducted regularly, have been shown to be more effective in groups rather than individually (https://pubmed.ncbi.nlm.nih.gov/34942157/). The social component of CST encourages shared attention, conversation, and peer engagement, which amplifies its therapeutic impact. However, dementia patients in rural or underserved areas often lack access to local CST groups and may need to travel long distances to participate in sessions.

We also wanted to raise awareness for the incredibly difficult work that many caregivers go through, drawing inspiration from the stories shared in communities like r/dementia, where family members openly describe the emotional exhaustion, financial strain, role reversal, and constant uncertainty that come with caring for a loved one with cognitive decline—often while balancing jobs, parenting, and their own mental health, with little formal support. We noticed positive support of AI assisted therapy for dementia patients by caregivers on the posted link and felt that a project like ours would be appropriate method for alleviating some of the stress that caregivers go through.

What it does

Patronum is an intelligent platform that enables dementia patients to participate in simulated group CST sessions with humanlike AI participants and a moderator. Designed as a supplement to in-person therapy, it allows patients to engage in more frequent sessions at their own convenience. It has 4 main features:

Simulated Group Sessions: An AI moderator guides structured CST activities with up to 5 AI participants and the patient, preserving the social dynamics critical to CST's effectiveness. To put it simply, we emulate a group therapy session with AI agents where patients are asked mentally stimulating questions that relate to reminiscent and sensory experiences rather than factual recall. For example:
- What do you miss from your childhood?
- What brings you back to your favorite memories?
Photo-Based Memory Stimulation: Sessions incorporate family-uploaded photos, prompting participants to describe, interpret, and react to visual cues to strengthen recall and communication skills.
Caregiver Dashboard: Caregivers can assign sessions, upload photos, and view an analytics dashboard with session summaries, engagement metrics, memory performance trends, and social interaction data. We provide research backed statistics that caregivers can refer to the information we provide to track the progress of their patients.
Cognitive Games: After a session, the patient is presented with a cognitive game that is related to the session (i.e. questions like who asked certain questions). We employ Error-Less Learning in these games, ensuring that patients have immediate access to the correct answers to the questions after answering them. This is done to ensure that patients have a relatively guilt-free environment.

Additional features

Session summaries written in plain language so caregivers can quickly understand how the patient engaged, without reviewing full transcripts

Automated alerts when engagement or performance drops significantly across sessions

Adjustable session length and difficulty to better match disease stage and patient stamina

How we built it

Our tech stack combines modern tools for a robust, scalable solution:

Frontend: Built with React for a responsive, intuitive interface. Complex state management handles real-time session data and large analytics datasets seamlessly.
Backend: Handles session orchestration, user management, and data persistence with a focus on low-latency performance across concurrent AI participants.
AI Processing: Powered by Claude, ElevenLabs, and HeyGen, multiple AI agents run in real time with optimized prompting to keep conversations natural and responsive. HeyGen allows for realistic appearing agents, while ElevenLabs powers the agents’ speech.
Real-time Updates: WebSocket connections ensure live session updates and smooth caregiver dashboard experiences.

Challenges we ran into

Prompt Optimization; We struggled with keeping agents speaking in a natural way, making the conversation feel realistic.
Safety: We needed quite a lot of testing to ensure that the AI wouldn’t go past the guardrails we set.
Meaningful Metrics: We did lots of research into what insights would be actionable for the caretaker but also generatable by AI.

Accomplishments we’re proud of

Building a realistic group CST experience that preserves the social interaction central to its effectiveness
Creating caregiver analytics that surface meaningful trends instead of raw data
Demonstrating how generative AI can support evidence-based healthcare interventions rather than replace human care

What we learned

Advanced video processing techniques in the browser
Real-time data handling with WebSocket connections to handle real-time updates effectively
AI model optimization for edge cases
Complex state management in React applications, especially when dealing with large datasets
Integration of multiple third-party services
The importance of user experience in security applications

What’s next

Future enhancements we're planning: Multiple language support
Integration with existing healthcare systems to facilitate session assignment by healthcare providers
Wider variety of conversation topics and questions available
Data encryption to protect patient privacy
Implementation of multiple papers used to protect against AI hallucination and malfunctions