According to the National Council on Aging, every 11 seconds, an older adult is treated in the emergency room for falling, and every 19 minutes, an older adult dies from a fall. A third of Americans over the age of 65 accidentally fall each year, and this is estimated to cost more than $67.7 billion by 2020. We wanted to make an IoT solution for the elderly, taking advantage of the Google Cloud Platform to be able to detect these falls more easily, and more quickly bring an emergency response to an elderly person who has suffered a fall.

What it does

Fallen is a security system that continuously analyzes its environment through a delayed video stream for people who may have fallen. Every 2 second frame is sent over to our Node.js server, which uploads it to Google Cloud Storage, and is then sent through Google Cloud Vision, which returns a set of features, that we filter. These features are passed to our own machine learning classifier to determine if the frame depicts a fall or not. If there is a fall, the system alerts all emergency contacts and sends an audio clip requesting help.

How we built it

We have a Node.js server that monitors the image feed coming in, connecting to Google Cloud Storage and Google Cloud Vision, and a Flask server that provides the machine learning classifier. We used the Android Things development kit to build a cheap monitoring system that will take a continuous stream of images, which is sent to the Node.js server, uploaded to Google Cloud Storage, and passed in Google Cloud Vision to retrieve a set of features which we filter based off of how relevant it is to distinguishing between a fall and not, namely LABEL_DETECTION and SAFE_SEARCH_DETECTION. These features are normalized and passed to the Flask server to classify whether it is a fall or not, and this response is sent back to the Node server. We used the Twilio API so that if a fall was detected, Twilio gives the emergency contact a call with an audio clip requesting for help.

Challenges we ran into

The Android Things camera cable was unreliable and unstable, and not able to stably provide a stream of images.

Built With

Share this project: