For those who have a loved one fighting dementia, the sensation of hopelessness to their forgetful condition often leaves the caretaker with frustration and anger. Dementia leads to a person's inability to communicate emotion and inability to recognize faces and names - crippling the most basic communication. We sought to innovate our way out of this disease's effects.
What it does
We were able to create a demo to showcase the underlying technology behind our idea. Using Microsoft Cognitive Services, we created a facial recognition model on video data that identifies and queries data on the person in the shot. The information is shown on our web application, but ideally, we would be using Augmented Reality headsets that have descriptive popup tags next to the person's face.
How we built it
We trained a model using Microsoft Cognitive Services API, used React for front-end, MySQL as a database, backend on Python/Django.
Challenges we ran into
We pivoted our idea at least 5 times in the first 24 hours and started development on a vast number of APIs and frameworks we didn't use. Additionally, we were not able to get our hands on an AR headset so we thought to use a Gear VR and access the phone camera, but we did not have a compatible phone among us.
Accomplishments that we're proud of
We are proud of surviving multiple existential moments and collaborating with one another. Our experience at Big Red Hacks extended further than intense development, and we are grateful to have attended this incredible event.
What we learned
Our front-end developer learned React from scratch for this entire hackathon (true beast), and we had two noob programmers who learned how to work with various APIs, web dev, and how to use Git for collaborative projects.
What's next for RecoSys
RecogSys has great potential to solve a critical problem associated with mental illness. While our focus is on solving mental health issues, we think this kind of recognition could be vastly applied to the common user. Recruiters, who cycle through hundreds of applicants a week, for example, could use this kind of recognition to keep track of names/qualifications. Teachers or professors with large amounts of students could identify their students, the period, and the class that they share as well. The possibilities are endless. We'd like to open source the software once we develop it for an AR lens and gain as much feedback as possible, and pitch it to NGOs who take care of these patients.