Inspiration: Somewhere, an Olympic athlete steps onto a $200,000 motion-capture rig, surrounded by biomechanists reading their every joint angle in real time. A few miles away, an elderly woman loses her balance on a staircase — no one saw the decline coming, because no one was measuring. A factory worker's back gives out after years of silent strain. A patient abandons their post-op physio sheet because there's no one to tell them if they're even doing it right. 1.71 billion people live with musculoskeletal disorders — the #1 cause of disability worldwide — and almost none of them have access to what elite athletes take for granted. We asked: what if the only hardware you needed was the phone already in your pocket?

What It Does: PhysioTwin turns any smartphone camera into a clinical-grade biomechanics lab. Stand in front of it for 30 seconds — a sit-to-stand, a squat, a simple movement test — and it extracts 33 3D skeletal landmarks, no wearable required. That data builds a living 3D Digital Twin, a glowing anatomical model that lights up exactly which joints are under strain. Behind it, a 6-axis Capability Profile (mobility, stability, movement quality, cardio, recovery, reserve) tracks how your body changes over time. Ask it questions and it talks back — a conversational twin grounded in your own movement history. Tell it you're planning a run, and its predictive engine simulates the toll before you take a single step. It even reads MRI reports and translates medical jargon into plain guidance. All of this, on a $50 phone.

How We Built It: We made a deliberate bet: zero hardware, zero compromise. MediaPipe pose estimation runs entirely on-device via WebAssembly at 30+ FPS — no video ever leaves the phone, only sanitized joint coordinates. Those coordinates feed a mathematical biomechanical engine computing flexion, asymmetry, and angular velocity, with CUSUM change-point detection catching sub-clinical deterioration before it becomes a symptom. For reasoning, we paired FastAPI with Groq-hosted Llama 3.1 for sub-second clinical inference, and gave the twin a body using Three.js — a WebGL anatomical model with real-time strain heatmaps. The whole thing ships as an installable PWA that runs on low-bandwidth connections, because equity was the point from day one.

Challenges We Ran Into: Going wearable-free meant fighting for every ounce of precision from vision alone — matching lab-grade signal with nothing but a camera. Explainability was its own mountain: a joint-risk score means nothing if a user can't understand why the system is worried. And every design decision ran into the same wall — how do you give someone real, actionable insight without ever crossing the line into diagnosis?

Accomplishments We're Proud Of: We built a system that needs literally nothing but a phone camera to produce clinical-grade movement analysis. We're proud that our privacy model isn't a footnote — it's architectural: video never leaves the device, full stop. And we're proud that one engine flexes from an elite athlete optimizing performance to a grandmother trying not to fall, without feeling like two different products.

What We Learned: We learned a phone camera can carry far more biomechanical truth than we assumed going in. We learned that in health-adjacent tools, trust isn't a nice-to-have — it's the whole product, and it lives or dies on whether the system can explain itself. And we learned that designing for radically different users at once — athlete, elder, rural patient, clinician — doesn't dilute a product. It forces every feature to earn its place.

What's Next for PhysioTwin: Contactless heart rate and HRV estimation straight from camera (rPPG), full gait analysis, and multilingual voice guidance in the next few months. After that: clinical trials with orthopedic institutions, EHR/FHIR interoperability, and a B2B rollout that puts a diagnostic-grade dashboard in the hands of physiotherapists managing 100+ remote patients at once. The end goal hasn't changed since day one — healthcare isn't a luxury, it's a right, and the future of physical medicine should already be sitting in everyone's pocket.

Built With

Share this project:

Updates

Submission history