Simple Play

Swordplay
Obstacle Course (Start)

Inspiration

Our inspiration were the games that many of us used to play in our childhood: Wii fit, Wii Sports and Wii Sports Resort! So we thought about if we could recreate anything from these games within 24 hours that would be really fun to play. We figured it out! Sword fighting from Wii Sports Resort was one of the funnest game modes some members of our team have ever played, so our group decided to try and recreate that first, and try adding other modes to spice up the game!

What it does

Our game uses body motions to play two different modes of minigames. Sword fighting and surviving an obstacle course!

In order for a good gameplay experience, the players will have to stand pretty far back from the cameras.

How we built it

We used python’s cvzone to send information about a player’s body over to Unity in order to make our players swing their sword, move left and right, duck and jump!

Challenges we ran into

The first challenge we had to get by was how we were going to get the sword to swing. The original plan was to have the sword follow a point on our hand (using hand tracking, instead of body movement) that Python will send over to Unity, and make the sword rotate and such as we swing our hands. The good thing about this is that our attempt worked! With one little issue…okay it was pretty big, it was the lag.

The way Python tracks your hand, if you swing your hand too fast then the program won’t be able to detect where your hand is for some time. But there was a solution! We used body tracking instead. When Python tracks a moving body, there’s almost no delay when it comes to sending the points over to Unity for calculation. So we got our sword to swing! All that was left was determining where and when to swing it. But that was also a problem that took a lot of math to resolve…but it was resolved!

The second challenge was how we were going to have cvzone detect two bodies. Our first thought was, “Google it”. And the first answer we saw was how difficult that was to accomplish, and the whole solution looked very hard to do especially when everyone on the team wants to sleep. So! The work around with this was to get another camera, and use cvzone to capture one body from the laptop camera (cam 1) and another body from another camera (cam 2). That way, we can send both different body positions over to Unity for sword fighting! (Although in this case we’re really only using our hand data from our bodies). However, there was one last small problem. When we first tested this using a samsung phone, our plan worked! Have the laptop camera pick up player 1, and the phone camera pick up player 2. The issue occurred when the laptop camera and phone camera saw both players. For some reason, both cameras only started tracking a single person. To get around this problem, we decided to keep our players separate, and have our players look at different screens to see their gameplay!

The third challenge was completing the obstacle course using body movement. This had a similar problem to our first challenge, our player moved way too slow because it was taking so long for Python to send over the exact points for the player. Fortunately, there was a work around this. Instead of sending the exact body points over to Unity, instead we decided that it would be best to determine whether our player was leaning left or right. We determined this in Python by seeing if our right hand is closer to our right foot than our left hand is to our left foot. Since when you lean one direction, that hand moves closer than your other hand. The player’s slow movement was fixed!

The fourth challenge was getting our player to jump in the obstacle course. The way it worked is that we track the distance between our player’s hip and their left ankle to determine if they’re bending their legs first, then once they release, they jump! However, this method comes with flaws. For starters, the jump distance changes depending on how far back a player is standing. If the player is too far back, the “bend” piece of code will already run, and there’s no way to get our player to jump unless they walk forward. To get around this, we just tampered with how low you have to go before jumping.

The fifth challenge was testing. Now granted, that’s every programmer’s challenge. However, for this project it was even more challenging. Normally when testing code, you run it, go through it, see if anything breaks, and get back to coding. The thing with this is that usually that programmer gets to sit down and watch their code. To test our code, we had to repeatedly move away from the camera and perform specific actions and if something broke then we had to fix it quickly and then move away again. We had to repeatedly set up our camera to make sure body motion was picking up what it was supposed to. There was a lot of exercise when it came to testing the jump function in particular (one of us was going to become a master of squatting soon). It was somewhat of an issue to test our code because we were in a pretty small space. But through perseverance, we eventually got our project done!

The final challenge was making our game look pretty and well…look like a game! We had to add music, some nice color and animations. Everyone on the team seems to be pretty satisfied with the final result.

Accomplishments that we're proud of

We were SUPER excited when we finally got the sword to swing. This was the part that we were all very worried we couldn’t get working, but we got it!

Another thing we were really proud of was getting cvzone to track two bodies and sending that information to Unity! At first glance it seemed impossible but our team pulled through!

The obstacle course mode was also another nice accomplishment for us.

Restarting a game mode after it’s completed wasn’t as stressful as we thought it would be, so that was a huge plus!

We also managed to finish our project while getting a good night’s sleep, which is a pretty big flex.

What we learned

We learned how when it comes to delay, cvzone body tracking seems to be far superior to cvzone hand tracking.

Some of us learned python hand and body tracking for the first time.

This game can really get you exercising. Swinging a sword and jumping as much as we did was somewhat of a workout.

What's next for Simple Play

More game modes!

The team was thinking about adding a bowling mode, since bowling’s super fun. A Simon Says mode was originally planned but had to be scrapped due to time constraints. Better and more accurate movement could also be accomplished.