Inspiration
Growing up playing competitive soccer, I broke my foot at a young age and spent months in physical therapy. Most of my recovery happened outside the clinic, alone, trying to remember how each exercise was supposed to feel. And I quickly realized I was not alone. More than 50 million Americans go through physical therapy each year, most of them doing the majority of their rehab without a therapist watching every rep. I constantly questioned whether my form was correct or if I was slowing my own recovery. That experience stuck with me. Physical therapy often fails not because the exercises are ineffective, but because feedback disappears once you leave the clinic. I wish I had this Kinetix back then, as it would have offered real-time guidance, restored my confidence, and turned my rehab from a guessing game into something structured, safe, and effective.
What it does
We built Kinetix, an AI-powered physical therapy and training coach that actively corrects your form in real time using computer vision and joint-angle analysis, automatically tracking reps and holds while comparing every movement to ideal biomechanics. The experience is fully immersive: with a Meta Quest headset, users receive live feedback from an AI agent directly in front of their eyes while they perform movements–, meaning they don’t have to check their phone, guess the correct form, or waste reps. It is like having a physical therapist or trainer with you at all times. This is precision rehab and training, delivered instantly, intuitively, and at scale.
How we built it
Frontend: Built with Next.js 13+, TypeScript, and Tailwind CSS to deliver a fast, responsive, real-time exercise interface that works seamlessly with live video Backend & Database: Used Supabase for authentication, workout plans, exercises, and structured biomechanical threshold data Pose & Biomechanics Engine: Implemented real-time pose tracking with MediaPipe, extracting joint angles and comparing them against custom threshold datasets for each exercise Real-Time Voice Feedback: Used LiveKit for low-latency, real time communication so users receive instant coaching cues during movement VR Integration: Enabled hands-free feedback through Meta Platforms Quest headsets, placing live guidance directly in the user’s field of view Speech Generation: Powered natural, motivating voice output with ElevenLabs for an immersive coaching experience Agent Orchestration: Used Overshoot AI to coordinate multiple AI systems at the same time, so they work together reliably
Challenges we ran into
There were numerous setup issues with Unity and the Meta Quest plugins, a necessary technology for running VR applications. These were mainly installation errors and long install times due to file sizes in the tens of gigabytes. Integrating computer vision with the right voice agent was challenging. We experienced problems such as the agent failing to speak, not evaluating form, and not accurately reporting results from the computer vision system
Accomplishments that we're proud of
Built a real-time AI physical therapy coach that analyzes live webcam video and evaluates exercise form using joint-angle biomechanics Implemented a custom threshold-based movement system to accurately track and validate physical therapy exercises Delivered instant corrective feedback during exercises, not after, enabling safer and more effective rehab Integrated VR support (Meta Quest) to provide hands-free, in-headset feedback directly in the user’s field of view Conversational AI voice agent that makes the product immersive
What we learned
How to apply pose detection and computer vision to interpret human movement in real time, including designing meaningful thresholds. Building and orchestrating a complex live AI agent, where different triggers, states, and user actions require different responses. Using LiveKit to power real-time agents capable of delivering instant feedback during active movement. Managing real-time data streams and synchronizing video, pose data, and AI responses. Integrating multiple AI and infrastructure services into a single, cohesive system. Developing applications in virtual reality using Meta Quest
What's next for flowers and joy
We hope to add the following features to this product to support the physical therapy use case.
Scaling Clinically and at Home: Tools for physical therapists to remotely monitor progress and adjust programs Structured recovery plans that bridge in-clinic care with at-home execution Secure data sharing between patients and providers Recovery Analytics & Progress Tracking Detailed exercise history, rep quality, and consistency tracking over time Visual recovery timelines to measure improvement and identify plateaus Data-driven insights that help users and therapists adjust programs intelligently We aim to sell Kinetix as a $250/month SaaS product to facilities. With ~38,000 PT clinics in the US, that’s a $114M TAM from clinics alone. Including sports training centers, schools with athletic programs, and colleges expands the conservative US TAM to ~$330M. Excluding slower sales cycles from schools, our initial SAM is $105M, and capturing ~3% (1000 facilities) represents a ~$3M ARR opportunity. Tables are included in the Devpost to show how we got these figures.
Log in or sign up for Devpost to join the conversation.