Inspiration

Five years ago, one of our close friend's grandfather had a stroke. The hospital part ended quickly. The hard part came after: months of repetitive exercises at home, alone, with no way to tell if they were working and no one watching between appointments. Reaching for a point, again. Making a fist, again. It is exhausting, it is boring, and most days nothing seems to change.

We kept asking one question: what if every repetition looked like progress the moment you did it?

An asterism is a small, recognizable star pattern inside a bigger constellation, like the Big Dipper inside Ursa Major. In Asterism, each session is one asterism, and together they build a patient's whole sky of recovery.

What it does

Asterism turns a laptop webcam into a rehab partner. No extra hardware.

You are Orion. Your shoulders and hands become stars on a night sky. The camera feed itself is never shown.
Every reach casts a spell. On your turn you pick a constellation and touch each star with your fingertip. Damage only lands when the whole constellation is drawn, so every rep counts.
Attack or block, like a PokÃ©mon battle. Blocking is one slow fist held for three seconds, then a wide open hand: real grip practice disguised as a shield. Enemies attack, guard or heal back, across a ladder of seven battles.
It stays safe. Calibration measures your comfortable reach, and stars never sit beyond it (Easy 80%, Normal 95%, Hard up to 110%). If you lean with your body instead of reaching with your arm, the star turns red until you sit tall again.
It measures what therapists care about, every session: active reach, accuracy, path straightness, tremor while holding, slowdown from fatigue, trunk leaning, grip holds and how long the hand takes to open.
It asks how you feel. After each battle: pain, tiredness and mood. Reporting a lot of pain tells you to stop and contact your therapist.
It gives tips after every move and a report after every battle, aimed at your weakest area.
Save files, like Minecraft worlds. Continue an old run or start a fresh one.
Coaches follow along from anywhere. Patients sign up inside the game; coaches sign up on the website, set themselves open or closed, and patients can only link to open coaches. Coaches watch session replays (the game screen, so it shows movement as stars, never the patient's face or home), see trends and write feedback that appears back in the game.

How we built it

The game is Python: MediaPipe for hand and pose tracking, OpenCV for rendering the night sky, NumPy for the movement math and Pygame for generated sound. Game logic is kept separate from drawing, so every rule is unit tested (183 tests).
Rehab measures come from the fingertip path: straightness is the straight-line distance divided by the path actually travelled, tremor is the wobble around a straight-line fit while holding on a star, and slowdown compares the last third of a session with the first.
Accounts and data live in Supabase (Postgres, Auth and Storage). Row-level security in the database itself makes sure a patient only sees their own data and a coach only sees linked patients. The database even refuses links to closed coaches. Each battle and its video upload in the background, and the game keeps working offline.
The website is Next.js, React, TypeScript, Tailwind CSS and Framer Motion, deployed on Vercel: a landing page plus the coach and patient portal.
A simulation of two bot patients plays the real game logic frame by frame for 16 simulated days, so we could test the reports before anyone sat in front of a camera.

Challenges we ran into

Lag. One enemy's shadow effect blurred the whole screen every frame and cost 54 ms. Rewriting it to touch only a small patch brought a battle frame down to about 5 ms. Then we found macOS was oversleeping by about 10 ms every frame, so we built a pacer that learns the delay and corrects for it: a steady 30 FPS.
Our measurements lied at first. The simulation showed an improving patient getting shakier. A faster, more confident arrival on a star was being counted as tremor. We now measure wobble around the hand's smooth drift. Without the simulation we would have shipped a metric that punished progress.
Safety versus spacing. Pushing stars apart so they don't overlap could push them past a patient's safe reach. Clamping afterwards pulled them back on top of each other. We solved both rules together.
Balance. Once blocking became a turn choice, the last battles became unwinnable for every simulated patient. For a rehab game, losing over and over is worse than useless, so we retuned damage until the early battles are winnable and the final boss is a real challenge.
Cloud permissions. Getting Supabase's row-level security, grants and storage rules exactly right, so a coach sees their patients and nothing else.

Accomplishments that we're proud of

A full loop: a patient plays at home, the battle uploads by itself, and a coach anywhere watches the replay and sends feedback that shows up in the game.
Safety built into the rules, not bolted on: the reach limit, the lean guard and the pain check-in.
Privacy by design: the coach sees movement, never the patient's face, and the database enforces who sees what.

What we learned

The hardest part of rehab software isn't tracking; it's measuring honestly. A metric can look great and still mean the opposite of what you think.
Simulated patients are a powerful way to test reports and game balance before real people are involved.
Small details matter for this audience: big text, no timers, every turn waits for the player, and losing still counts as exercise.

What's next for Asterism

Testing with real stroke survivors and physical therapists, and tuning our thresholds with them.
A one-click installer for the game.
More exercises (wrist rotation, two-handed reaches) as new spells.
Meeting health-data requirements (such as HIPAA) before real patient use.

Asterism supports home practice; it is not a medical device and doesn't replace a therapist.