Critical Leap

• 

  Gameplay

Inspiration

Especially with the development of technology and social media, many people spend hours sitting in the same chair. Worsened with bad posture or unoptimal chairs, problems like chronic back pain and lessened hip mobility can often arise. Other side effects include high blood pressure and an increased risk for diabetes. Thus, Critical Leap aims to encourage movement and blood flow after long periods of sitting, using a fun game experience as an incentive to do physical movement.

What it does

Critical leap is formatted as an endless runner game, where a player-controlled character travels along a path. Along the path are obstacles that require the in-game spirte to jump, but in order to make this sprite jump, the human has to kangaroo jump. This motion is tracked by the program, and upon detecting the jump using the webcam, the program triggers the in-game sprite to jump. The challenge is to travel as far as you can along the path, which is only possible if you time each jump carefully and you yourself (the physical YOU) don't run out of energy!

How we built it

Critical leap was built using Panda3D in Python. This 3D engine helps provide an immersive experience that 2D games would struggle with. This program also uses Google's Mediapipe Pose package to detect landmarks on humans. Each "landmark" (head, ankle, shoulder,...) in the video stream is located and assigned a pair of (x, y, z) values. By using these values, and initial calibration data, Critical Leap is able to detect when the user jumps.

Challenges we ran into

A challenge that this project faced was being able to have the game run smoothly while all the complex computations occured in the background. To achieve a relatively smooth gameplay, the game runs at fast framerate but the webcam and mediapipe pose detection only run once every five frames. Because of this, the timing of your jumps don't exactly align with the sprite's jumps. This is a problem that is especially apparent in the video demo, but no easy solutions exists (other than running Critical Leap on a faster computer).

Accomplishments that we're proud of

Although mostly for simplicity, Critical Leap acheives a clean, aesthetic, low-poly look that is completely original; all the models were built by hand in Blender. Despite it's simplicity, Critical Leap turned out to be very fun to play, it was like the Google Dinosaur game but YOU had to jump to make the sprite jump. We are proud of this level of interactivity we achieved and hope to extend this idea to other exercises in the future.

What we learned

Developing Critical Leap helped teach concepts like constructing (using Blender) and rendering 3D models inside a game. It also taught how to use Google's Mediapipe Pose, which was surprisingly simple as by calling detector.detect(), you would get a list of 31 points, referring to 31 different positions on the body.

What's next for Critical Leap

Critical Leap is only a proof-of-concept, additional mechanisms like daily streaks, global leaderboards, and custom maps can be added to further incentivize user participation. Also, rather than being confined to only kangaroo jumps, Critical Leap can expand to detecting exercises like push-ups or squats and gamify them.

Built With

• cv2
• mediapipe
• panda3d
• python