Inspiration
Every 11 seconds, an older adult is treated in the emergency room for a fall & every 19 minutes, an older adult dies from a fall. While these accidents are often unavoidable, we wanted to build an emergency system to help notify authorities quicker and perhaps save lives. This not only helps with providing mental security to the elderly, but also helps empower them to live more autonomous lives into their later years.
What it does
FallSafe is a machine learning service that integrates with a live video feed (Nest Cam) to detect when someone has fallen down inside a home, and then alert authorities and emergency contacts if there is an incident.
How we built it
We have a Flask web server that serves as a portal to interact with FallSafe. The web app has a UI to create an account, register cameras, and see the live stream / detection probabilities. The backend is a makeshift Nest API that pulls images from the camera every 2 seconds & processes them via Matroid, a TensorFlow wrapper to determine whether anyone has fallen in the screen image. If our system determines there is a high likelihood that someone has fallen, the authorities will be alerted via the SafeTrek api and emergency contacts are notified via the Twilio API - a picture of the incident is also sent to all emergency contacts.
Challenges we ran into
We had expected the Nest API to be easier to use, but they rate limit the number of images you can pull per minute and the livestream had no easy way to retrieve still images. We therefore hacked our own Nest API by writing some automation scripts to scrape the live web footage from a browser, post those images to Amazon S3, and then consume our own images through our Flask backend. This ended up working out well for us, as we were able to reduce the size of each image significantly using the python image library optimization package, which allowed us to run more computations and display images more quickly.
Accomplishments that we're proud of
We were able to create something pretty cool, and the machine learning worked surprisingly well - it can detect fallen people readily and doesn't get fooled by much.
What we learned
We learned that you should always overestimate on deliverables - there are a lot of problems that you don't foresee, and so you have to plan for the most unexpected things.
What's next for FallSafe
This seems like an MVP for a Nest software update that perhaps can help save lives, so if this gets enough attention maybe Nest (or now Google) will pick up the project / re-implement it!
Built With
- amazon-web-services
- d3.js
- flask
- matroid
- safetrek-api
- twilio
Log in or sign up for Devpost to join the conversation.