Each of us was coming with different experience, but we were all interested in machine learning. After forming our group we attended the tensorflow.js Gear Up workshop together and got interested in Teachable Machine and ML5's PosNet. We knew we wanted to try out these tools so we spent some time brainstorming after the workshop to devise the best project we could create given our skillsets and time constraints.
We landed on Fall Safe!
What it does
Fallsafe uses real-time video monitoring to estimate and classify poses using PosNet and a trained Teachable Machine neural network. We classified poses into two categories: Safe or Fallen. If the user has fallen and stayed down, we send out an alert.
How we built it
We followed along with YoutTube videos and blog posts about ml5 projects and PosNet to get the framework of what we wanted to do. We then trained our neural network on teachable machine using a limited dataset of recorded video snapshots and images from datasets. Finally we integrated our neural network into the model, created and formatted a Flask web server to host our neural network, and integrated them together..
Challenges we ran into
Accomplishments that we're proud of
What we learned
There are MANY ways to integrate machine learning into code and these days it's incredibly accessible. Even from 2 years ago the resources have grown considerably to learn about and deploy these technologies.
What's next for Fall Safe
Our demo shows the potential for integrations of this type to be widely distributed within homes of vulnerable communities. Before that can happen, we would need to ensure our model was trained on a larger dataset across many different locations with many different people so that it will be robust and helpful for anyone who wants to use it. As shown in our demo, the neural network is still hesitant to say that someone has fallen over. This can be solved by using big datasets to train it in all body positions, including some where the person is partially off-camera. As the internet of things within the home becomes more popular, cameras will be able to send your google home or smart assistant of choice the alert that you have fallen down. Your smart assistant can then access your contacts, verbally ask if you're alright, or execute another specified safety protocol. If our validity and reliability becomes consistent enough to eliminate the considerable risk of false alarms, we can think about offering direct access to emergency services.