Ever since I got to know how powerful transfer learning is when used with a lite model like MobileNet I wanted to automate tasks using gestures captured through webcam. One does not need to buy costly controllers to play games anymore. Using ml5js I was quickly able to do the same and have a great time building and playing it.
What it does
User at first trains the pre trained MobileNet model to recognize the exact gestures for left, right and no movement of the student and then enjoy playing the game by making the same gestures for the corresponding movements and dodging all the incoming calculators from different directions,speed and sizes. Highscore is tracked and updated.
How I built it
Using p5js online editor I used ml5js to do transfer learning on MobileNet and there by control the motion by webcam input. Multiple rounds of hit and trial method to try and get the best look and feel of the game possible. Assets like sound, images, fonts where all taken from opensource and licence free sources. Code was pushed to github and github pages was used to quickly get the game up and running.
Challenges I ran into
The formatting and deciding layout for the same where a bit tricky as well as the the point of collision between calculator and student took a couple of iterations to get it right.
Accomplishments that I'm proud of
I am really proud of this project and to be able to complete it in time.
What I learned
A lot of new features and functions in p5js and about Github pages.
What's next for Calcy Dodger!
Make it a multiplayer and have a backend to keep track of scores of multiple players.