PlayFit-AI

PlayFit-AI interface
game interface

Inspiration

Many people spend hours gaming but struggle to stay physically active. Living in a country with high obesity rates and heavy video game usage, we asked: what if gaming itself could become the workout? PlayFit AI was inspired by the idea that exercise should feel engaging, not forced, by meeting users where they already are, playing games.

What it does

PlayFit AI transforms physical movement into real-time game controls. Using AI-based pose estimation through a standard webcam, the system detects full-body movements like punches, kicks, and jumps, and maps them directly into gameplay actions. This turns traditionally sedentary gaming into an active, full-body exercise experience, no gym or equipment required.

How we built it

We built PlayFit AI using a real-time computer vision pipeline. A pre-trained pose estimation model detects body landmarks from a webcam feed. On top of this, we implemented an AI control layer that interprets movements, filters noise, and maps detected poses to in-game actions. The system runs in real time and interfaces directly with games through virtual input controls, creating a responsive and immersive experience.

Challenges we ran into

One of the main challenges was balancing responsiveness with accuracy. Small delays or false detections can break immersion and make gameplay frustrating. We also had to carefully design movement thresholds so actions felt natural while still encouraging meaningful physical effort. Ensuring stability across different body types, camera setups, and lighting conditions was another key challenge.

Accomplishments that we're proud of

We’re proud of building a working, real-time prototype that successfully turns physical movement into gameplay without specialized hardware. The system demonstrates that AI-driven pose estimation can be used not just for tracking, but for meaningful, interactive control. Most importantly, it proves that exercise can be seamlessly integrated into activities people already enjoy.

What we learned

We learned that effective AI products don’t always require training massive models from scratch, thoughtful integration of existing ML models with strong system design can create powerful experiences. We also learned how critical user experience, latency, and feedback loops are in real-time AI applications, especially when humans are part of the control system.

What's next for PlayFit AI

Next, we plan to integrate a lightweight PyTorch-based action recognition model to personalize movement detection for individual users and reduce reliance on fixed rules. We also want to add fitness metrics like session duration and intensity, expand support to more game genres, and explore applications in home fitness, physical therapy, and active learning.