OnTrack

OnTrack gives people the confidence to take control of their recovery. With just a phone camera, we show clear progress, so patients don’t guess if they’re getting better, they can see it.

Comment

Inspiration

When everything already feels stacked against you, the hardest part of recovery isn’t just the pain...it’s the uncertainty of not knowing whether any of it is actually working. We were moved by what we witnessed: our own family members enduring physical therapy day after day, pushing through self-doubt, held back by fragmented and unreliable care that left too much to chance.

OnTrack was built to give those moments something to stand on. It helps dejected physical therapy patients visualize progress and build momentum throughout their recovery journey.

Key Features

  1. Body-Tracking with Pose Overlay - To set an example of a proper physical therapy exercise for the patient

  2. Pose Analysis and Voice Correction - Recording measurements / angles using the user's body-tracked skeleton, and outputting helpful vocal feedback based set thresholds.

  3. Risk Prediction - Using user's movement scores to predict re-injury risk, and fall prevention risk (recovering patients are at most risk)

  4. Recovery Roadmap / User Data - Tracking improvement in the user's range of motion scores, while processing user information: PT documentation, user & clinician goals, demographics, user movement scores on a longitudinal basis.

How we built it

Two parts: a React Native client for onboarding, live pose capture, ghost coaching, and on-device analysis; and a Python/FastAPI service that receives session exports (summaries + CSVs + landmarks), persists them in Postgres (Alembic migrations), and runs Fetch.ai uAgents plus Chroma/RAG to produce reports, progress views, and tailored advice the app reads after each visit.

Challenges we ran into

  • Rendering the motion / animation of an accurate pose in a way visually helpful and clinically accurate way was incredibly difficult and took several runs.
  • Reliably detecting the boundaries of one rep of an exercise from raw pose landmarks, since the correct signal varies by exercise type (e.g hip descent works for standard squats but fails for single-leg squats, where knee flexion occurs with minimal hip movement)

Accomplishments that we're proud of

  • Consistently testing with MediaPipe (and tons of good squats) to figure out the the right thresholds, rep mechanisms (finding what movements constitute one rep)
  • Setting a standard protocol for unifying historical, live, and clinical patient information to map at-home, consistent patient progress.
  • Providing an accessible platform for movement tracking; existing platforms use expensive sensors, and subscription fees. Ours only requires a phone camera!

What we learned

We learned how clinical intent has to be translated into engineering choices: which signals you trust, what you smooth versus what you preserve, and how much uncertainty you expose to the user versus hide behind a simple score. We also learned that integration beats isolation. The product only “clicked” when capture, coaching, export, and read‑back were wired as one loop, and that for movement apps, perceived reliability (stable visuals, consistent reps, predictable uploads) often matters more to users than marginal gains in a single offline metric.

What's next for onTrack

  • A planning agent that generalizes this tool for any Physical Therapy program, and researches/creates a pipeline for users with specific injuries / goals. It is able to generate thresholds to assess accuracy of poses, in specific.
  • Apply the longitudinal data framework for other avenues involving specific movement data such as tracking neurological disorders.
  • Groundsource of truth for clinical literature on its own

Built With

Share this project:

Updates