NeuraFlow

The AI glasses for dementia patients that you'll never forget.

Comment

Inspiration

As all of us have come across Alzheimer's or dementia so far, we understand how difficult it is to lead a life in that way. When we looked at our family members or friends that deal with these problems, we were determined to use our time at TreeHacks to tackle this - and allow it to truly enhance people's everyday lives. Could we build Meta Glasses or something similar that can truly change the lives of these individuals? We are leaving this Hackathon knowing we just did this.

What it does

The NeuroFlow glasses provide dementia patients with an AI Healthcare Assistant that can provide them with any information about things or people they want to remember. It measures the heart rate of the patient, and if it notices distress, then it will ask the dementia patient how it can help. It has full context awareness of what the user likes for them to store on their behalf. The dementia patient can then easily recall whatever they are struggling to remember. With our implementation, it can even read and respond in 100+ languages!

How we built it

The hardware component of the product are 2 ESP32s that provide camera data and heart rate information. This data is analyzed to determine if an individual's heart rate is abnormal - in such a case we expect the dementia patient to be in need of help. This is when we activate our classification model that determines who a person is, providing helpful supplementary information to allow people to work. In the event that the dementia patient says, "Flow", our agent turns on to ask the user what they need help with. Access to their entire day's worth of information (and the internet too!) is at their disposal. In the event that they need some calm nature music, they can get that too!

Challenges we ran into

Some challenges we ran into were primarily related to connection. Learning how to parse the data provided by the ESP32 consistently with the WiFi faltering and our own novelty with the subject was especially difficult, but over time, we loved every minute of the challenges, and we learned so much! We also were trying to implement a CNN at first, but classifying many images of us through the grainy camera would have been difficult. To improve our efficiency, we applied FaceNet instead using embeddings.

Accomplishments that we're proud of

Within 36 hours, we were able to build this entire project from 0 to 1. We started with the three of us and 2 ESP32, and we are leaving the Packard building with a solid voice recognition model for dementia patients that can not only register when they are stressed, but also recognize people around them and provide them with information that they need stored.

What we learned

That we can be a lot more ambitious nowadays. Each of us learned from each other - from the hardware to AI architecture to voice recognition and response models. We managed git conflicts, learned how to work under intense time pressure, and even how to CAD a sweet design.

What's next for NeuraFlow

We see ourselves scaling up our compute more, so this tool can be more assistive to more people with alzheimer's and dementia. We genuinely believe in this tool, and we think we can be used by many people that not only have dementia but could use a healthcare assistant at any given time. It truly can enhance human lives at a very minimal cost (literally fractions of a cent). We can also pick up on more biomarkers to be even more confident about how to respond with the in-built agent. Memory agents to parse through our vast amounts of data to personalize the user experience will also be a crucial next step.

Built With

Share this project:

Updates