TheraPlay

TheraPlay helps people move better by turning rehab and fitness into an engaging, AI-powered experience. Using real-time pose tracking directly in your browser, it gives instant coaching cues.

Comment

Project Story

Inspiration

We wanted to make physical therapy and fitness more engaging, accessible, and privacy-friendly. Many rehab apps require sign-ins, uploads, or clunky software — we thought, what if real-time form coaching could happen directly in your browser, with no data leaving your device?

We also noticed how hard it is to keep kids motivated during rehab. So we added playful, game-like modes that make moving fun again.

What it does

TheraPlay is an AI-powered form coach that runs fully in the browser. It uses real-time pose tracking to guide users through exercises with instant cues like “hips up,” “stack elbows,” or “align knees.”

  • For adults, it provides coaching, rep counters, and hold timers.
  • For kids, it turns rehab into games — like “Flappy Face” (head tilts flap a bird) and “Tower Balance” (hold steady to keep a tower upright).
  • It’s privacy-first: no video is uploaded, everything stays on your device.

How we built it

  • Frontend: HTML, CSS, JavaScript (custom UI + responsive design).
  • AI/ML: MediaPipe Pose for landmark detection.
  • Game mechanics: lightweight JavaScript mini-games integrated with pose data.
  • Deployment: Runs in-browser, mobile-ready, no install required.

Challenges we ran into

  • Making pose tracking smooth across laptops and mobile devices.
  • Designing a UI that works for both adults and kids — clear for rehab but playful enough for younger users.
  • Balancing accuracy vs. performance so it runs well without needing extra hardware.

Accomplishments we’re proud of

  • Built a working prototype that coaches form live in the browser.
  • Integrated fun kid-friendly modes to make rehab less intimidating.
  • Achieved 100% on-device processing — users keep full control of their data.

What we learned

  • How to integrate pose estimation into interactive experiences.
  • The importance of designing for accessibility and engagement, not just accuracy.
  • That gamification really does help turn something like rehab into something people want to keep doing.

What’s next for TheraPlay

  • Add more exercises and adaptive difficulty.
  • Explore accessibility features for patients with limited mobility.
  • Partner with healthcare professionals to validate and refine the coaching cues.

Built With

  • HTML / CSS / JavaScript
  • MediaPipe Pose
  • Browser APIs (Webcam, Canvas)
  • Responsive web design

creators: Anthony sinchi, Salssabil El Jbari, Sofia Cobo, Adesh

Share this project:

Updates