TtoppouziGO!

Inspiration

Inspired by the emergence of XR appications we wanted to develop something similar. We decided to embrace the experimental nature of Augmented Reality applications and employ various features to both enhance our learning and most importantly provide a full-fledged experience for the user to enjoy.

What it does

Our application realises real-time combat between two users. Each user can enact a sequence of actions to play during their turn, executing a variety of attacks (e.g. normal attacks, heavy attacks or quick attacks) or perform actions such as drinking potions to boost their health. We applied tried and tested game mechanics to provide a 'balanced' and fair game-play experience and we went a bit further and added armour overlayed on top of each character.

How we built it

We used a pre-trained network called PoseNet that accurately maps reference points on a human figure (e.g. hips, wrists etc.) and we then developed our own model based on it, for the poses we wanted to use using ml5.js. As such each player's move is a successful sequence of poses. Using the play library of p5.js we implemented sprites for the armour pieces that follow each player's limb movements. Our game is hosted on Netlify with all game information stored on Firebase.

Challenges we ran into

When we moved from the online p5.js web editor to implementing on a website we experienced a significant performance drop on the responsiveness of the recognition algorithm. This hinders classification of poses and subsequently the recognition of sequences. However, we are hopeful as this runs perfectly on the p5 web editor!

Accomplishments that I'm proud of

We are proud of tracking our joints and building a model that recognises our poses with great confidence and that we managed, with zero background knowledge to apply visual effects on a live video feed.

What we learned

No impressive app can be completely polished overnight!

What's next for Kombatto

We aim to finish bringing it all together under a hosting service and forwarding the link for friends and qurious gamers to test and provide feedback.