Every single coder suffers from "cant-stop-wont-stop", "how did is it 8 pm already?" coding sprees but breaks are needed, and here's a fun game to help us with it. Given that we are now spending more and more time in front of the screen it is important that we take care of ourselves with 5-minute breaks. PoseMan Gamify this
What it does
Through a game of combined hangman meets pose, this game helps the play move around for a few minutes. The game takes in webcam feed passes it to an ML model which transposes an animated SVG onto the screen. The user needs to use his body to Guess letters for an object shown on the screen before time runs out.
How we built it
- A pre-trained Tensorflow.js neural network called Posenet is used for detection of poses and transposing an animated SVG.
- The second neural network was used for guessing the Words from poses. This required us to train models using images captured using the webcam. After this ML5.js library was used to make an interface with model and application.
- The entire application is bundled using parcel and a CI/CD pipeline to Netlify was set up which will automatically build and deploy app on pushing the staging branch.
Challenges we ran into
- Making 2 neural networks run together in a browser
- A game has a lot more moving components, keeping track of changes and thinking of proper designs was challenging.
- Bundling and deploying introduced other set of challenges
Accomplishments that we're proud of
- Bundling/minifying of application
- CI/CD using Netlify and GitHub
What we learned
- Async programming
- Machine Learning
- Tensorflow.JS and ML5.js
- Pretrained networks and training new models with output from another neural network
What's next for PoseMan
Multiplayer abilities so you can compete against your fellow coders, you code together, you pose together.