Our inspiration for Dotmentia came from one of our team member's late great-grandparent who suffered from dementia. Towards the end of his life, he had severe trouble even recognizing the voice or faces of those that lived with him every day. Close family members and family members were hurt by the realization that one of their favorite grandparents could not recognize who they were. As a result, this idea was born.
What it does
Dotmentia allows a dementia patient to ask a voice assistant such as Alexa or Google Home Mini the identity of a pre-registered person. This pre-registered person will have the model train on a previously uploaded image. Using a phone camera, the phone will recognize the person and send a message back to the Alexa or Google Home. The Alexa or Google Home will then report the identity of the person to the dementia patient. An additional feature is that the pictures will be stored locally on the device for the patient to use as future reference, if needed.
How we built it
Our project consisted of two parts — an app with facial recognition capabilities, which we used OpenCV and Android Studio to build. The second part of our app, which incorporated the Google Home mini, and voice commands integrated Google Cloud's Platform and DialogFlow.
Challenges we ran into
- We initially kept changing ideas as couldn’t agree on an idea, w got really tired (ZZzzzz) - Working with Android was very difficult and unfamiliar to us - There were a tons of bugs we and the mentors couldn’t resolve - We tried to use react native for the majority of the project but it didn’t work as intended - We couldn’t get the Google Home Mini to integrate with Android Studio
Accomplishments that we're proud of
- Getting a somewhat functional app and experimenting with Google Home Mini, despite having very limited success - Pulling an all-nighter and not falling asleep (2/4)
What we learned
Android Studio is a pain, facial recognition is a pain, know your limits
What's next for Dotmentia
Integrate the Google Home Mini with the Android App (we were unable to do this because we had a lot of issues regarding getting the correct environmental variables for Android Studio set up and even running in the first place.