Inspiration

Each of us was coming with different experience, but we were all interested in machine learning. After forming our group we attended the tensorflow.js Gear Up workshop together and got interested in Teachable Machine and ML5's PosNet. We knew we wanted to try out these tools so we spent some time brainstorming after the workshop to devise the best project we could create given our skillsets and time constraints.

We landed on Fall Safe!

What it does

Fallsafe uses real-time video monitoring to estimate and classify poses using PosNet and a trained Teachable Machine neural network. We classified poses into two categories: Safe or Fallen. If the user has fallen and stayed down, we send out an alert.

How we built it

We followed along with YoutTube videos and blog posts about ml5 projects and PosNet to get the framework of what we wanted to do. We then trained our neural network on teachable machine using a limited dataset of recorded video snapshots and images from datasets. Finally we integrated our neural network into the model, created and formatted a Flask web server to host our neural network, and integrated them together..

Challenges we ran into

We ran into some challenges with our initial plans for audio in Javascript. We were unfamiliar with many JavaScript concepts, and it took a lot of tinkering to finally get our alert mechanism to work. The neural network's weights had to be tweaked as well to optimize it's functionality.

Accomplishments that we're proud of

Learning! We all wanted to learn more about machine learning and we definitely got to do that while also getting to know each other and enjoy the experience. We also learned a lot about javascript.

What we learned

There are MANY ways to integrate machine learning into code and these days it's incredibly accessible. Even from 2 years ago the resources have grown considerably to learn about and deploy these technologies.

What's next for Fall Safe

Our demo shows the potential for integrations of this type to be widely distributed within homes of vulnerable communities. Before that can happen, we would need to ensure our model was trained on a larger dataset across many different locations with many different people so that it will be robust and helpful for anyone who wants to use it. As shown in our demo, the neural network is still hesitant to say that someone has fallen over. This can be solved by using big datasets to train it in all body positions, including some where the person is partially off-camera. As the internet of things within the home becomes more popular, cameras will be able to send your google home or smart assistant of choice the alert that you have fallen down. Your smart assistant can then access your contacts, verbally ask if you're alright, or execute another specified safety protocol. If our validity and reliability becomes consistent enough to eliminate the considerable risk of false alarms, we can think about offering direct access to emergency services.

Built With

Share this project:

Updates