-
-
Open hand - palm to the camera and tuck thumb
-
Closed fist - trap thumb
-
None
Inspiration
Whenever something bad happens in the public space, the consensus is 'What's the CCTV footage?'. We know the system is there and we rely on it in times of need. But this investigation usually happens after the fact. CCTV also has a role in prevention, and even if this is not something that the general public is used to, we can use the system to avoid many more unfortunate events. For example, we can allow the public to reach out for help in an easy way, without making unnatural movements like suddenly searching for a phone, rape alarm, pepper spray etc when feeling threatened.
What it does
The code as it is built right now is straightforward. It relies on recognising the signal for help in a video stream. When the code recognises a succession of open hand - palm to the camera and tuck thumb, followed by a closed fist - trap thumb, it prints out a message.
How we built it
I used the ipynb file from AndroidML-HandGestureRecognitionModelTraining from the rock-paper-scissor game provided by Google in Google Colab to get inspired and write the actual code. I used the Gesture Recognition from MediaPipe Studio to test the Model and lay out the foundation of the project.
Challenges we ran into
Most of the challenges were technical, where I either didn't know how to implement the idea or once I had the plan and found some useful resources, I couldn't connect to Google Drive or get the right path of the file. Another challenge was teaching the model to recognise the difference between a closed fist and a closed fist with the thumb trapped. This was easily solved by adding more images for the model to train on.
Accomplishments that we're proud of
This project in itself is what I am proud of. When I first enrolled on this hackathon, I came with lots of false narratives like it would be difficult to find the time to work on it, or it would be difficult to solve technical challenges, but all of these proved false. There are resources online to help you start, you just need to keep an open mind and start working on whatever project you want to. For the technical challenges, I couldn't believe how easy it is to work in Colab. I'm no Phyton expert, but this platform can explain the code to you, can tell you what your error is and how you could fix it.
My Resources
Coding: Rock-Paper-Scissors Model training with MediaPipe Model Maker - ML on Android with MediaPipe Series
Colab - get image from Drive: https://medium.com/data-arena/how-to-load-files-insert-images-in-google-colabatory-c5d1365d60d4
SOS - Image Inspiration: https://tbwa.com/work/signal-for-help/
What we learned
I learned that nowadays, once you have a plan in mind, it is much easier to implement it due to this amazing technology. I joined the challenge without being familiar with Google's AI and Machine Learning platforms. I don't even use Python in my day job, nor have I ever built a project using it. But tools are here to help. Only action is needed.
What's next for SOS Gesture Recognition
If this project will have a future, I would like to work with a team. More people bring various perspectives and ideas and this is something of infinite value. In my head, I would like this project to be scaled and implemented. I would like any person who needs help to be able to receive it in time. Is an easy gesture to do and with so many cameras around us, it is easy to pick up. In the future, once the gesture is picked up, the alert could contain the location of the camera, and maybe even have a separate line for a police AI to investigate the request and act if the situation requires.
Log in or sign up for Devpost to join the conversation.