For the past 11 months, the world has been in quarantine, and many people have switched from in-person learning and meetings to online classes and conferences. However, distance learning and online conferences present multiple core issues that make the experience less intuitive and more stressful and frustrating. I built the Simple Human Action Processing Engine for Videos (S.H.A.P.E Videos) to solve these issues and create a more natural online conferencing experience.
What it does
S.H.A.P.E Videos recognizes hand gestures using OpenCV and produces a specific output per hand gesture. On a video conferencing call, instead of looking for the unmute button during a call and losing your train of thought, you would show a simple hand gesture to the camera, and then the program would increase the volume, raise your hand, mute, unmute and other things, creating a more natural experience for your online video call.
How I built it
I coded my project in Python and used libraries such as OpenCV, NumPy, imutils, and scikit-learn. I used OpenCV for video capturing and manipulation, NumPy for data manipulation, and imutils/scikit learn for useful functions.
Challenges I ran into
The most challenging part of the project was detecting how many fingers were shown from a segmented hand image. I had multiple ideas on how to do this and had to try and fail multiple times before I got a method that worked.
Accomplishments that I’m proud of
- Learning what was necessary for the programs as contours were complicated to me at first
- Comprehending what I learned and turning that knowledge into a cohesive program
- Building a successful program that detects hand gestures and helps the online learning experience.
What I learned
I learned to perform complicated video transformations using NumPy and OpenCV.
What's next for S.H.A.P.E Videos
- More types of hand gesture tracking
- Improve accuracy
- Add real-time ROI refreshing