Inspiration
- Our themes are social welfare and health and wellness.
- The social welfare part of it is the fact that you can play games with people if you want therefore allowing you to socialize with people while exercising.
- The health and wellness part of it is because you're exercising as a lot of kids don’t exercise because they want to play video games by themselves or friends.
- American children aren’t getting enough exercise. The CDC reports that over 50% of adolescents from 12 to 15 are physically unfit.
- We love playing games, and we want to make exercise fun.
What it does
We coded a program that lets the player “play” games by jumping, squatting, moving left/right. Using the point coordinates we can determine if they are jumping, squatting, leaning left/right, or joining hands. These actions are used to “press” keys. We use math to get the proper distances and manipulate the key presses.
How we built it
We use mediapipe to get the poses from the camera. Then from the pose we get specifc x and y values of body parts such as your wrist, shoulder, etc . After this we calculate the midpoint of these values to find out which quadrant of the screen you are on. We press the corresponding keys. There is a huge library of games compatible with our program as any game that uses keys on your keyboard work It is easy to start just launch your favorite game and the program to start We plan to make this a app in the future
Challenges we ran into
Originally we planned to integrate mouse movements into this, but the model was having trouble recognizing hand movements One of the main libraries in this app Pyautogui (used to simulate keypresses) was not working properly, and we had to create multiple virtual environments to make it work
Accomplishments that we're proud of
We learned how mediapipe works in a day!! We learnt about tkinter, and user frames.
What we learned
We learned about ML, and how to create a GUI with python.
What's next for PlayConnect
We plan to make this project into a app so more people can benefit from exercising. We wanted to design a UI and adapt more games to this code but we ran out of time. While fitness games already exist, such as RingFit Adventure, we plan to make this app 100% free. Implement mouse
Youtube link: https://www.youtube.com/watch?v=eeChf5lnzvA
Log in or sign up for Devpost to join the conversation.