We set out to create an application that would be a valuable resource for individuals with disabilities, such as Alzheimer's or low vision. Our aim is to give people the tools necessary to recall and revisit their past experiences. Individuals with Alzheimer's disease typically struggle with memory loss, so our goal is to jog their memory. For individuals with low vision, we want to create a framework that enables them to revisit their memories and "see" the bigger picture.
What it does
In its final form, Retrospection will have an online and mobile app presence. In its simplest form, users have the option of choosing to take a picture or record a segment of audio with a time stamp. Images get processed and pushed through Clarifai to generate tags that describe what is included in the image. As tags are generated the information is grouped into a database entry where it waits for Amazon Alexa to pull the tags out of the database. Audio recording entries are passed through speech to text; and then text is incorporated into the Amazon Alexa skill. Once Alexa has been updated with new entries, users can ask Alexa about a specific day/time and Alexa will announce either the tags from that instance, or highlights from a audio recording.
How we built it
We began by creating a Wordpress website and connecting to Microsoft Azure. From there we implemented an android application utilizing Clarifai in Android Studio. We also created a skill for Amazon Alexa.
Challenges we ran into
Several challenges occurred as a result of combining different technologies to create Retrospection. Connecting Amazon Alexa with a blob database in Microsoft Azure proved to be a difficult task because we were unfamiliar with the coding languages necessary to implement these connections.
Accomplishments that we're proud of
In the relatively short time of the Hackathon and as first-time hackers, we were able to envision a project with incredible possibilities. Although we could not finish the full implementation, we are very excited about our progress and proof that siblings can work together to solve a problem (eventually).
What we learned
We learned how to utilize API's, Microsoft Azure, Java/Android Studio, Amazon Alexa for the first time. Sleep deprivation leads to accidentally inventing a new word for your app name!
What's next for Retrospection
We plan to continue working on the implementation/ database connect problems. We would really like the app, website, and Alexa to be seamless. We are also going to be on the lookout for additional cases/uses of this application.