Gesture Recognition - Convolutional Neural Networks

Inspiration

Wanted to recognize gestures to automate mundane tasks

What it does

Recognizes hand gestures using the Webcam. Each of those gestures are mapped to a specific function (Jump, Duck etc.) and those functions are automated using py auto gui. This is used to control the Google T-Rex game.

How I built it

The OpenCv Libraries contain filters to remove background noise. After the noise has been removed, the image is saved. Then it proceeds to predict which gesture it is using the CNN Model (CNN's perform exceptionally well in image and video processing). The CNN model is inspired from the MNIST model. Used Keras and OpenCv in Python and py auto gui was used to automate the spacebar/arrow keys

Challenges I ran into

For this to work Good Lighting conditions were needed. I did not have much knowledge on how to implement a CNN and initially was lost on how to get started It has got a low accuracy of 70% There is also a time lag of image processing which takes around 5 nanoseconds

Accomplishments that I'm proud of

Atleast got started and the progress was something I never expected it to be

What I learned

About CNN's and Python

Built With

• keras
• mnistdataset
• python