Our inspiration came from being dissatisfied with standard keyboard controls while playing video games.

What it does

Our program takes in unique keyboard strokes inputted by the user and, using machine learning technology, trains the program to remember and recognize those keyboard strokes so that players can program custom controls into their video gaming.

How we built it

We built the program by implementing neural networks with the Gluon API, an interface for defining and training deep learning technology. We mapped keyboard events to unicode values, that were then put into an array and passed to the neural network for training. Afterwards, we tested our program for functionality and accuracy on a simple program that we coded. The program moves a ball upon receiving keyboard instructions after the user trains the program to recognize the keyboard commands.

Accomplishments that we're proud of

We are proud that we were able to implement all the parts of the program successfully (and most importantly the neural network) so that the ball in the sample game that we created was able to move upon learned keyboard inputs.

What we learned

Throughout the process, we learned how to implement a neural network using the Gluon API as well as discovering all the subtleties that are vital to this process. We also learned how to map keyboard events to unicode values and then input those values and manipulate them in an array in order to process them successfully for training a neural network.

What's next for Keyboard Gesture Recognition

Keyboard gesture recognition can be used to make video gaming faster, more intuitive and more fun for gamers and can be applied to any computer programs to create custom keyboard shortcuts after the implementation is optimized and more functionality is added.

Built With

Share this project: