Z - Fighter: Born to Fight

Inspiration

Due to the atrocious global pandemic of Covid-19, all of us were forced to stay indoors during the lockdown. Soon we all became habitual to the comfort of living in our homes, lying down 24/7 binge watching Netflix shows and eating junk food, which lead our fitness down the drain. During such stressful times it was hardly possible for people to be motivated to workout. We are a team of tech enthusiasts, who saw this as an opportunity to use the latest technology of Augmented Reality to help people get fit in a fun way!

What it does

Ever wondered how cool it would be to actually experience being your favorite game character? Heads up! This is no longer just a fantasy. A new warrior has entered the ring and this time it is you! To make this fantasy into reality, we came up with the concept of Z-Fighter, a Snapchat Lens game. Z-Fighter offers very exciting and immersive Gameplay to fans of Dragon Ball Z and similar fighting games, as they can now jump into the action and have an instant familiarity with the intuitive controls. Be it kicks, punches, boxing, Hadouken or Kamehameha, all of these moves can be performed by the players in real life. Z-Fighter uses Full body tracking Model to track user's body key-points and later classify their movements into Game Actions, using Neural Networks, to have immersive fighting experience. Based on the user’s movements, the enemy will be attacked or the enemy’s attack would be blocked. This allows people to fight the enemy on their just instead of using a keyboard, they use their own fists! If the user is victorious, they also get to flex their triumph by sharing their power level with their friends. In today's world with increasing importance of physical health, this game will also encourage players to perform exercise while having fun at the same time. So see you in the Ring!

How we built it

We used the Full Body Tracking model of Lens studio to track the movements of the user. These movements were mapped to the First person view player in the game. The enemy in the game has certain hit-points which will be active for an interval of few seconds. The users will have to hit that hit-point before the time is out else the enemy will attack them. This was achieved using 3-D distance approximation where we calculated the distance vector from the users’ fist to the hit-point and if the fist reaches a certain proximity of the hit-point, it gets triggered and the enemy is attacked. If the user misses, the enemy attacks and the user can squat to block the attack. The squat motion is achieved by tracking the downward motion of the pelvis of the user. The 3-D model of the enemy was made using Blender and its fight animations were defined and developed there which were later imported into Lens Studio. The Power Level feature uses the Face tracking feature of Lens Studio. Several JS were programmed in order to trigger events such as animation of the enemy, based upon the movements of the user. On top of that, Audacity was used to generate music effects and audio editing.

Challenges we ran into

The biggest challenge was to map the user’s movements to the player’s fighting actions in the game. It was also problematic to decide the UI of the game as we wanted to give the users the most natural and immersive experiencing game. Building the enemy, Broly, 3D model was also challenging especially while making its animations. To overcome these challenges we programmed JS Scripts such that they will help calculate the proximity of the hand and target and event triggers if it goes below a threshold. The UI was well thought out and successfully developed such that it felt realistic to the users and lastly we harnessed the power of Blender to tackle all 3D model challenges.

Accomplishments that we're proud of

Our Lens was one of the 20 Lenses that were selected Globally in the official Snap AR Creator Residency Program where we got the opportunity to develop this lens under the guidance of Snap Inc’s creative and engineering teams and earned the title of Snap AR Partner (then Official Lens Creator). We have over 1 million plays of our lens on Snapchat.

What we learned

Our learnings vary from a lot of technical learning to behavioral learning too. Developing this lens made gave us a much broader overview of Lens Studio and how powerful of a tool it is to develop AR experiences. We also learned to collaborate and network with other engineers from different countries and learn from them.