Telekinesis

Cover Page
Game Select Screen
Reach, for Duchenne muscular dystrophy Lift each arm out to the side and up to the light. Tests range of motion of shoulders
Research behind Reach
Tap - For Parkinson's disease Tracks speed, rhythm, and whether taps shrink or slow down.
Research behind Tap
Floor Rise - for Duchenne muscular dystrophy. Times each stage and how long the hands push on the legs, a classic sign of Duchenne.
Floor Rise 2
Research for Floor Rise

Inspiration

Five years ago I tore the labrum in my shoulder, and I was picked to try a new physiotherapy exercise. To track my range of motion and recovery, I had to go back to the doctor every time. Each visit was expensive and took hours, and the doctor spent a few minutes watching me lift my arm and measuring with a protractor. I could have done that at home.

Clinical trials work the same way, at a much bigger scale. Treatments for conditions like Duchenne muscular dystrophy and Parkinson's are judged on a few in-clinic motor tests spread over a year. Each one is a single attempt on a single day, and getting there is hard on the people the trial is meant to help.

So I built the app for that. Telekinesis.

What it does

Telekinesis turns standard clinical motor tests into short games you play at home with a webcam. Each session produces objective measurements that keep a trial participant's digital twin up to date.

We use Google's MediaPipe pose model, which runs entirely in the browser. On every frame it finds 33 points on the body (shoulders, elbows, wrists, hips, knees, ankles), and a separate hand model finds 21 points on each hand. The video goes into the model and onto the screen and nowhere else. The games only ever receive these points, and each game turns them into a measurement using simple geometry.

There are four games, each based on an assessment trials already use:

Rise is the 30-second chair stand. It counts full stands and the time each one takes, and records rises that sink back separately. Floor rise is the timed rise from the floor in the North Star Ambulatory Assessment, which was the primary endpoint in the vamorolone trial for Duchenne. Besides the time, it detects a Gowers' manoeuvre (pushing up on the thighs) and shows where in the rise the participant slowed. Reach measures the highest each shoulder lifts the arm out to the side, left and right kept separate. This is the measurement I used to travel to a clinic for. Tap is the finger-tapping item of the MDS-UPDRS exam for Parkinson's. It turns a 0–4 score given by eye into taps per second, rhythm consistency and how much the taps shrink over time.

How we built it

React 19 + TypeScript + Vite, deployed on Vercel.
MediaPipe Tasks Vision runs Pose and Hand landmark models in the browser. Video goes into the model and onto the screen and nowhere else. Only the landmarks reach the games. Measurements, not footage. Each game is a small state machine that takes one pose frame at a time and outputs measurements. Thresholds are relative to the participant's own body (shoulder widths, arm length, hand length), so a game reads the same however tall someone is or wherever they stand.
A simulated participant moves a synthetic skeleton through each game at a realistic pace, including a partial rise and a Gowers'-style floor rise. The games can be demoed without a camera, and the same body drives the test suite (60 Vitest tests).
A canvas renderer draws the thermal view: desaturated video, a cool body blur, and per-joint glows blended with screen.

Challenges we ran into

Scale-free measurement from a single 2D camera. Pixels mean nothing when every participant stands at a different distance. Each game needed a ruler that stays fixed through the movement. Shoulder width barely changes between sitting on the floor and standing, which made it the ruler for Floor rise.

Telling a bad score from a bad recording. This mattered most for trial use. Quality is tracked per frame, stored separately from the measurements, and never alters the score.

Accomplishments that we're proud of

Measurements a trial would actually use, such as failed rises, Gowers' detection, per-side shoulder range and tap decrement, instead of a single game score.

What we learned

Regulators have clear expectations for remote measurements (the FDA's guidance on digital health technologies). Recording quality and data provenance matter as much as the measurement.

What's next for Telekinesis

Live digital twins. The study team's twin chart uses mock data right now. Next is fitting each participant's twin from their own baseline sessions.

Validation. Run the games alongside in-clinic NSAA and MDS-UPDRS assessments to measure agreement, which is the step that comes before any exploratory endpoint.