ReMind

We have developed an AI companion using LLMs to help improve the quality of life for people with dementia and their caregivers.

Comment

Inspiration

Dementia is a leading neurodegenerative disorder affecting over 55 million people globally. Fading memory comes as a great personal loss and frustration for those affected AND their caregivers. Large Language Model (LLM) such as ChatGPT by OpenAI has made it plausible to tackle some of the problems surrounding dementia patients and their caregivers. Our prototype called ReMind represents a significant step forward in this direction.

What it does

ReMind is geared to improve the emotional well-being and independence of people with dementia and their caregivers by 1) Initiating adaptive conversations with the patient and creating a positive emotional atmosphere 2) helping them remember important information, like names of their loved ones. And, 3) providing feedback to caregivers on their interactions with the patient.

How we built it

We employ GPT-4 to analyze conversations and maintain the patient’s subjective history database, accessible only to the patient. We also extract emotions expressed in a conversation using “Arousal” and “Valence” of each sentence. AI companion is aware of the current emotional state of both the patient and the caregiver and can respond differentially to either of them and can adapt. A working prototype in Python using OpenAI can be found on the link below.

Challenges we ran into

We had to implement a history database in order to persist information across GPT-4 sessions. We also had to programmatically implement and integrate speech-to-text and text-to-speech pipelines with GPT-4 to address ease of interaction.

Accomplishments that we're proud of

We implemented an end-to-end working prototype of ReMind, especially using direct speech input, personal history management, and a way to evaluate emotion in ongoing conversations.

What we learned

GPT-4 is a powerful tool that can not only synthesize an arbitrary text but can also generate highly intelligent conversations in any given context. It can also be used for deep analysis of any text.

What's next for ReMind

Implement speaker identification to create a history database specific for each speaker, allowing for personalized interaction with GPT-4 across speakers. Develop flexible strategies for creating input prompts to get more conversationally natural and helpful responses.

Built With

Share this project:

Updates