The inspiration for the memory hub comes from seeing many patients with dementia struggle to keep track of their daily routines, and seeing that many of their relatives and friends get worried over the personal safety of the patients. We want to create a space for patients with dementia to record their daily routines. Memory hub will not only serve as a gentle everyday reminder for patients with memory loss conditions but also help the relatives and friends of the patients be more aware of the emotions and behaviors of the patients.
What it does
We integrated speech recognition, sentiment analysis, and topic modeling to let patients record and share their daily activities with the people who care deeply about them. Then we use the Twitter API to make these posts available to the general public.
How we built it
We built the frontend using react + material UI. We built the backend using Django, AWS; S3, Twitter API, and EC2. We used the ML libraries from AWS such as Comprehend and Transcribe.
Challenges we ran into
We had trouble recording the current location and time of the user, but we fixed that using google Geocode. We also had trouble synchronize the recorded audio file and the record in the database. We solved the problem using POST to the database when we are downloading the data from the user's end.
Accomplishments that we're proud of
We are proud that we have a fully functional pipeline that achieved our original goal of helping people with recording their day to day life.
What we learned
We learned a lot with respect to team coordination, and the communication between the frontend and backend development
What's next for Memory Hub
We intend to have more coverage on memory hub in terms of better ml models and better speech analysis.