BOXercise

Alan fighting a virtual boxer
Ethan fighting a virtual boxer
Fighting Vincent inside the game
Timon preparing to fight a virtual boxer
Facial detection functioning
Fist detection functioning

Inspiration

We sought to combine the excitement of virtual gaming with the benefits of physical exercise. Wanting to challenge ourselves with a computer-vision-based project, we designed a virtual boxing environment that seamlessly integrates real-life movements into the digital realm.

What it does

BOXercise is a virtual boxing environment that uses computer vision to translate the user's movements into the game. The user can choose their difficulty level and spar with a virtual opponent by physically moving side to side and throwing real punches. It provides both a moderate workout and an engaging gaming session.

How we built it

BOXercise is built using the pygame framework. It uses a pre-trained opencv model for facial positioning and mediapipe for hand keypoint detection.

Challenges we ran into

None of us had any significant computer vision experience, so getting punch detection and facial positioning to work consistently and accurately proved an interesting challenge. Multithreading, especially involving a camera, was also challenging to implement, albeit essential to the performance of our project.

Accomplishments that we're proud of

We are proud to have successfully built an efficient computer vision system using unfamiliar libraries such as opencv and mediapipe. It remains consistent and fast enough to allow for an enjoyable gaming/exercising experience.

What we learned

We learned how to make games using pygame, and how to use computer vision for human body tracking, which can make virtual applications more immersive and encourage physical activity.

What's next for BOXercise

Given more time, we would like to implement more gameplay features using CV (kicking, 2D movement, full-body tracking). Player customization in the form of skins or a leveling system would also make the game more engaging. Online multiplayer is also a feature we'd like to add, as exercising with friends can often be more enjoyable.