Machine Learning Game Example(Hard to Handle)

For the Treasure Hacks Hackathon, we decided to create a game the uses hand gestures as player inputs. In order to do this we decided to use Google's Teachable Machine to recognize hand gestures, and the Python Pygame Library in order to actually create the game. First, we trained the Teachable Machine to recognize hand gestures pointing upwards, to each side, and a flat palm. Then we created a mini platforming game using Pygame. We decided to name it Hard to Handle and proceed to get to work. The final game is a simple platformer that just has a mini hand jump up randomly generated platforms while collecting coins. If you get to a certain score there will also be a surprise. (you may want to end the program at this time) In order to connect the game to Teachable Machine, we created a JavaScript file that runs the teachable machine, then returns the input a python file. The python file takes the input from the JS program and turns it into a keystroke which the game then reads. Unfortunately, we ran into some problems when we tried to get the game to read the inputs from the JS code. While the keystrokes would appear in a text file, the game would not recognize them. Currently, the project does not fully work, but the game can still be played using the arrow keys. If we had more time, we might have solved this issue, however, we recognize that time constraints are a major part of hackathons, and we will try to do better next time.

Built With

• javascript
• node.js
• pygame
• python
• tensorflow