Patronus - A spell against dementia

Inspiration

Dementia is a disease that affects the individual but also everyone around them. They are often not given the care they need because they need it full time. We wanted to help those with dementia and give them a little bit of themselves back. Basically, to use AI to be more human.

What it does

Patronus is an App that interacts with the individuals directly. It answers any questions they may have and remind them of things they may have forgotten. It also identifies people around them and provides a video memory to help them recollect who they maybe interacting with. It is highly personalized and learns about the user with every interaction.

How we built it

We used a qdrant vector data store to store personalized information about the use and built a RAG model on top of it. The RAG model is powered by the powerful Gemini LLM. We integrated the twelve labs' multimodal AI API for video querying via text and image. The rich UI is powered by React Native while the powerful backend is hosted on a Flask server.

Challenges we ran into

Integrating multiple APIs and DBs proved to be a challenge. Lot of trial and error was needed to perfect the LLM prompts to prevent dangerous misinformation and make our AI responsible.

Accomplishments that we're proud of

Created the entire app from start to finish in less than 24 hours due to technical difficulties. A App that we are proud of that will help people in need.

What we learned

We learnt a lot of these different APIs and how generative AI is not just a gimmick.

What's next for Patronus - A spell against dementia

Distribute it to the people in need. Add personalised text to speech in the sound of their loved ones. Doctor integration to provide further context on medication reminders. The possibilities are endless

Built With

• flask
• gemini
• javascript
• langchain
• mongodb
• python
• qdrant
• react-native
• twelvelabs