Being in the shoes of caregiver's shoes of patients with Alzheimer's disease was our main motivation for this product. Grandparents of our teammate David are suffering from this disease, so lowering the burden of patients and caregivers had real life implications.
What it does
It is a vr-based personalized memory training app that helps to recall important memories of patients with Alzheimer's disease. How it works is: before the last stage of Alzheimer, family members of the patient will scan the place of an event and record the conversations that took place in that event. Once the patient puts on the VR set that's playing our app, he/she is back to the major event that took place and HeyAlz replays back the event by helping reconstruct the memory.
How we built it
We first used an outside app called Polycam to render the place of memory in 3d. Then in a local folder of an iphone, we would select the file that we want the patient to experience and post that file into Google Firebase Storage. After that we used Google API (Speech-to-Text Transition) to record the voice of the surrounding and convert that to text so that patients putting on the VR headset could not only hear the conversation that took place in the setting of memory, but could also see it visually. Lastly, we made it so that the IOS frontend and Unity communicates the string converted from speech to display that text in VR space.
Challenges we ran into
A lot of the problems we ran into was technical mostly because we tried dealing with some technologies that we've never handled and had experience with.
Accomplishments that we're proud of
Could come up with specific model of app and implement it with VR and sound effects.
What we learned
We learned that this model has a diverse and endless application and room for improvement.
What's next for Hey Alz
So we didn't get to finish implementing trimming of the recorded voice to prompt patients to guess what conversation took place in the place of memory. And so if we have more time, that's definitely something we'll work more on.