Amazon Echo is a wireless speaker and voice command device designed by Amazon. It was launched in the market in June 23, 2015. Amazon Echo is capable of recognize certain voice patterns and responds to them. Miao Zi and Jeffrey came out with the idea that if we could place an Amazon Echo in every ward of a hospital, then we could actually monitor patients' activities in the ward and respond to their needs when they send requests to Amazon Echo.

What it does

Collects commands/ requests from patients through Amazon Echo and then get data from Alexa, process the data and output the processed information to two platforms - website and an app. Our website displays graphs of frequencies of different requests from different ward (here we assume each ward has one Amazon Echo) while our app is a platform that sends push notifications of patients requests to nurses/ doctors of the hospitals.

How we built it

We used Amazon Echo to get patients' requests and then through Alexa, we got their requests and then saved the information in firebase. Then we used react.js to do the following data processing:

i) Classify different requests into 5 categories: water, restroom, walk, medicine, emergency.

ii) Create graphs of frequencies of emergency requests ( restroom, medicine, emergency) and normal requests(water, walk) of different wards.

After we were done with the data processing, we posted the graph to the website and different requests to our app. We aims to help doctors/ nurses to do some analysis on patients' activities based on the graph and hopes that through the app the paramedics can better attend to patients' need.

Challenges we ran into

i) Learning how to create new skills for Alexa. None of us knew how create new skills for Alexa and it took us a while to learn it.

ii) Connecting Alexa to firebase. Since we need to process the information from Alexa and then distribute it to our website and app, we decided to store Alexa's output in firebase. However, there is any example about connecting Alexa to firebase so we have to figure that out ourselves. The biggest problem we have encountered was that Alexa's output was not sent to firebase before Alexa's sessions end which caused us to keep getting nothing in our output. We managed to use javascript's async lib to fix this problem.

iii) After connecting Alexa to firebase, we encountered another problem - processing data! We decided to use react.js to do data processing. However, none of us had any prior experience with react.js so we spent a lot of time figuring how to create graphs and extract required information from the data in our firebase.

Accomplishments that we're proud of

The thing we are most proud of this project is that we are creating two platforms that can be useful for paramedics and ease nurses/doctors burdens in handling patients' requests. Amazon echo is a pretty new device and we are really proud that we could work on this new device along with Alexa to use the sophisticated natural language processing techniques that Amazon has created to make a positive impact to the healthcare industry.

What we learned

i) Learn how Amazon echo and Alexa work and manage to extract information from the output that Alexa produces based on the input from Amazon echo

ii) Learn how to do basic information extraction such as creating graphs and sending useful notifications to hospital staff

iii) Team work distribution

What's next for Help, Alexa!

i) Getting feedback from people working in healthcare industry on types of information that they need and try infer the data they need based the patients' requests

ii) Currently we are classifying requests into five categories. We want to add more so that we can handle patients requests better. Also, we want try implementing real time response system through Alexa.

Share this project: