Recall - Cognitive Care

A gentle daily companion for memory care: Clara’s voice, family memories, meds, sleep, and caregiver peace of mind, built for Margaret and families, not to replace real care.

Comment

Inspiration

This project was inspired by the healthcare gap we have observed through our close family and friends. We have some grandparents who have suffered from neurodegenerative diseases, and still do. Every time we talk to them, we realize the fight against neurodegenerative disease is a steep uphill battle. When conducting more research on the treatments for these diseases, we found that oftentimes the research and computational theories don’t translate into more beneficial healthcare. This is why we built this app, deeply inspired by our frustration with the current healthcare system and its treatment of neurodegenerative patients, trying to address its shortcomings and gaps. Recall bridges this gap by creating a user-friendly mobile app that can be used by affected patients to help manage their daily lifestyle and improve their long-term outcomes under the supervision of a doctor who can monitor their progress.

What it does

Recall is a cognitive care companion for seniors living with dementia and memory disorders. It combines two experiences in one app: a calm, orientation-focused interface for patients, and a real-time care command center for caregivers and supervisors. For patients, Recall offers State Reconstruction. This is an AI-generated "right now" card that tells the patient who they are, where they are, and what time it is, which is refreshed every 5 minutes. Another main component is Clara, a voice companion they can speak to naturally for reassurance, schedule reminders, and memory support. There are also camera-verified medications. The patient can point their camera at a pill bottle and Recall confirms the right medication before you take it using underlying vision models. To exercise the patient’s mind, there are cognitive games. From Wordle-style puzzles, Sudoku, and Connections, the patient has many ways to stimulate their mind gently. In case of an episode where distress signals are detected from the patient either through their app navigation or conversations with Clara, Recall presents Comfort Mode. This is a full-screen intervention with breathing exercises, a calming narrative, and a caregiver call link. Emergency SOS is a hold-to-activate button that dials 911, texts emergency contacts, and alerts the caregiver dashboard instantly. These features allow for a simple and grounding experience where the patient is constantly on track with their daily life and does not lose track of who they are or where they are. On the other end, for caregivers and supervisors, Recall provides a live ACSE Score dashboard (Adaptive Cognitive Stability Engine). This is a 0–100 behavioral health score built from detected signals like perseveration, sundowning, disorientation, and missed medications. Additionally, Storm Radar is an hourly cognitive "weather forecast" that highlights high-risk times like the 4–8 PM sundowning window. Full medication adherence history, sleep logs, and a searchable event timeline are also there in the app to give the supervisor a better picture of where the patient is in their recovery timeline. There is also real-time cross-device sync so a caregiver on a laptop sees the patient's status update the moment something happens.

How we built it

Recall is a local-first React + TypeScript SPA built with Vite. It was styled with Tailwind CSS, and packaged for iOS via Capacitor. All patient data lives in the browser's IndexedDB via Dexie, so there is no cloud database, no account creation, and no patient data leaving the device. AI features are accessed through a Cloudflare Worker API proxy that keeps API keys server-side and enforces layered fallbacks: LLM: Cloudflare Workers AI (Llama 3.1 8B) → Groq API fallback Text-to-speech: ElevenLabs (eleven_turbo_v2_5 for Clara's voice) → Workers AI MeloTTS → browser speechSynthesis Vision: Groq Vision → Google Cloud Vision → manual confirmation The ACSE engine is a rule-based behavioral scoring system built entirely on the client side. It is used for tracking signals like semantic speech loops (Jaccard similarity ≥ 0.55), rapid tab switching, inactivity, re-dose attempts, and a 1.5× sundowning multiplier between 4–8 PM. Many research studies have shown this is a time where a lot of episodes happen. Cross-tab caregiver sync is achieved without any WebSocket server using the browser's BroadcastChannel API combined with localStorage. This means a caregiver can have the supervisor view open on a second device and see patient ACSE changes in real time. The home screen's globe was pre-rendered using an offscreen Three.js WebGL renderer (65 frames baked at startup) and driven by scroll position and by giving a rich 3D visual without any runtime GPU cost.

Challenges we ran into

One of the biggest challenges was designing technology for a population that may sometimes struggle with technology itself. Many dementia-focused applications assume users can easily navigate complex menus, remember instructions, and troubleshoot issues. We had to rethink nearly every interaction to minimize cognitive strain and cognitive load in order to create a user experience that feels calm, predictable, and reassuring. We believe we successfully addressed this challenge, as the app prioritizes simplicity, clear navigation, and low-friction interactions designed specifically for cognitively vulnerable users. Furthermore, creating the ACSE behavioral scoring system required extensive research into dementia symptoms and longitudinal patient studies. Research shows that behavioral symptoms such as agitation, disorientation, and sundowning are not random, but instead strongly correlated with disease progression and functional decline in Alzheimer’s and related dementias (Scarmeas et al.). Sundowning patterns, in particular, follow measurable circadian cycles, with increased confusion and agitation occurring in late afternoon and evening hours (Volicer et al.). Additionally, studies on continuous behavioral monitoring suggest that sleep disruption, activity changes, and agitation can be used as reliable indicators of cognitive and functional deterioration over time (Alberdi et al.). More recent machine learning research further demonstrates that multimodal behavioral signals can even be used to predict agitation episodes in advance with high accuracy (Jiang et al.). Turning these clinical and behavioral concepts into measurable digital signals was one of the most technically challenging aspects of the project, requiring us to translate real-world neurological patterns into a structured, rule-based scoring system. Scarmeas, Nikolaos, et al. “Behavioral Symptoms and the Progression of Alzheimer Disease.” Neurology, vol. 68, no. 23, 2007, pp. 1807–1813. PubMed, https://pubmed.ncbi.nlm.nih.gov/18071039/. Volicer, Ladislav, et al. “Sundowning and Circadian Patterns in Dementia Patients.” Journal of Geriatric Psychiatry and Neurology, 2001. PubMed, https://pubmed.ncbi.nlm.nih.gov/10841213/. Alberdi, Arantza, et al. “Smart Home-Based Monitoring of Alzheimer’s Disease: A Review of Data-Driven Approaches.” Journal of Alzheimer’s Disease, 2018. https://alz-journals.onlinelibrary.wiley.com/doi/10.1016/j.jalz.2018.02.004. Jiang, Y., et al. “Predicting Agitation in Dementia Patients Using Multimodal Sensor Data.” arXiv preprint, 2025. https://arxiv.org/abs/2506.06306.

Accomplishments that we're proud of

We are proud of building a solution that meaningfully supports both dementia patients and their caregivers within a single, unified system. Instead of focusing on just reminders or monitoring alone, Recall brings together memory assistance, emotional reassurance, cognitive engagement, and caregiver awareness in a way that feels cohesive and easy to use. One of our biggest accomplishments was creating Clara, an AI voice companion designed to provide calm, natural conversation for patients. Clara helps with orientation, reminders, and reassurance, offering support in moments of confusion or anxiety without feeling clinical or robotic. We also developed the Adaptive Cognitive Stability Engine (ACSE), a behavioral scoring system that translates complex patterns like disorientation, missed medications, and cognitive distress into a simple, understandable metric. This gives caregivers a clearer picture of patient wellbeing over time. We are proud of how we linked the demo patient’s sleep data with an Apple Watch, so the seamless transition is able to happen without any technical difficulties. Overall, we are just really proud of ourselves since this is the first app we created, and through group difficulties with communication and responsibility, we prevailed and our app turned out great! We realized somewhere along the way that no matter what happens at this hackathon, whether we place or not, we are all planning to give this app to some of our loved ones who are affected by neurodegenerative diseases.

What we learned

Through building Recall, we learned that designing for healthcare, especially for neurodegenerative conditions, is as much about empathy and human understanding as it is about engineering. Small design choices, like how information is phrased or how often it updates, can significantly affect whether a user feels calm, confused, or overwhelmed. Throughout this project, we always had to think from the perspective of an older person, trying to imagine their thoughts, sentiment, and how they would act or feel. We also gained hands-on experience working with modern web and mobile technologies, including React, TypeScript, and Capacitor. We learned how to structure a complex application using a local-first architecture. Working with IndexedDB and building a system that avoids reliance on a central database taught us how to design for privacy, reliability, and offline-first performance. On the AI side, we learned how to integrate multiple models and services with fallback systems to ensure reliability in real-world conditions. Balancing language models, voice synthesis, and vision tools helped us understand both the power and limitations of current AI systems.

What's next for Recall - Cognitive Care

Next, we want to expand Recall from a prototype into a more complete, clinically informed support platform for families and caregivers. A major focus will be improving personalization so the system can better adapt to each patient’s daily patterns, communication style, and cognitive baseline over time. We also plan to integrate wearable devices like biosensors to track signals like sleep quality, movement, and heart rate, which could help provide a more complete picture of cognitive and physical wellbeing. This goes further than just an Apple Watch by creating a complete ecosystem. This would allow Recall to detect subtle changes earlier and offer more meaningful insights to caregivers. On the caregiver side, we want to build a more advanced dashboard that helps families and healthcare professionals understand long-term trends, not just real-time status. This includes clearer timelines, pattern detection, and more actionable summaries that reduce the burden of constant monitoring.

Built With

Share this project:

Updates