MotionSurfer

Inspiration

Remembering our favorite game as children, Subway Surfer, we were inspired to transform the game into an immersive and fun experience that incorporates physical exercise and gaming. This led us to develop the idea of using computer vision motion tracking to power players to control movements not with a simple tap of a finger but rather with the full movement of their body.

What it does

This project uses computer vision libraries to map player movement to key presses to simulate a fully immersive gaming experience where you are the controller.

How we built it

We used OpenCV, CVZone, Mediapipe, and Pynput to realize this project. OpenCV and Mediapipe track player motion with CVZone tracking hand and pose. This input was then mapped accordingly to corresponding key strokes using Pynput.

Challenges we ran into

Player calibration is somewhat difficult and if done incorrectly, can cause inconsistencies in movement and gameplay Deciding how sensitive we wanted our program to be to player movements, while trying to strike a balance between responsiveness and accuracy

Accomplishments that we're proud of

Despite having very little CV knowledge, we managed to complete a working product that is genuinely enjoyable to play We were proud of how much attention our idea was able to attract from other groups, mentors, and judges Completing our first hackathon!

What we learned

Some python syntax; all of us had minimal experience in Python but through a lot of googling, reading and trial and error, we made it work A lot about computer vision and how to receive and process live video input in a program How to navigate obstacles as a team, especially when we disagreed about the best solution

What's next for MotionSurfer

Automate calibration using the box size relative to the frame size Train an ML model to learn to play the game, and integrate it as a bot player to play against Can create a model to play at the same skill level as you and tracks your improvement Expand our idea for more mobile games