Spotter

Inspiration

Every fitness app I've used is basically a notebook. You type in that you did five squats, and it believes you. It has no idea whether those were five good reps or five reps of something that's going to wreck your lower back in three years.

That's not a small niche of the market, either. 540 million people used fitness apps in 2025, driving about $3.4B in revenue, and almost all of that is logging — sets, reps, weights, entered by hand. The thing that actually knows the difference between a good rep and a bad one is a person standing next to you, watching. In a gym that person is called a spotter, and they cost $40 to $100 an hour, which is why most people don't have one.

So observation is the expensive thing in fitness, and it's rationed by price. Meanwhile roughly 608,584 resistance-training injuries reached US emergency departments between 2013 and 2022. Average patient age: 22.8. Over a third of them were trunk injuries. That's more or less exactly the person filming themselves alone with a phone leaned against a water bottle.

I didn't want to build another tracker. I wanted to build the watching.

What it does

You point your phone at yourself, or drop in a clip you already took. Spotter figures out which of five lifts you're doing — squat, push-up, lunge, deadlift, plank — from the shape of the movement itself, counts the reps, and then shows you what no rep counter shows you: the actual frames. Here's rep three. Here's the bottom of it, timestamped. Here's your knee angle across the whole set, and here's how long you spent going down versus coming up.

It writes a short review in plain language with timestamps attached, using six sampled keyframes plus the numbers it measured. It talks to you while you lift, so you don't have to look at a screen mid-set. It can replay the movement in 3D at your set's actual tempo. And it tells you, up front, what percentage of frames it could actually see your body in — because a review built on footage where you walked out of frame is worthless, and you should know that.

The analysis runs on your phone. Your video doesn't go anywhere. That started as a privacy decision and turned out to be the reason the whole thing can be free: inference happens on hardware I don't pay for, so my marginal cost per analysis is roughly zero. Most AI products get more expensive the more people use them. This one doesn't.

The part I actually care about

There's a version of this app that's much easier to build and much worse. Same video, same model, and then a big number on the screen: Form Score: 82%. People love that number. I built it. It looked fantastic.

Then I thought about what it actually was. A phone camera sees a flat projection of a body from one angle. It doesn't see your spine. It doesn't know your proportions, your history, or what your knees are supposed to do. Putting a percentage on top of that isn't measurement, it's a costume.

So I deleted it, and replaced it with a rule I applied to every layer: nothing can claim more than the layer underneath it can prove.

MediaPipe measures deterministic 2D geometry on-device — angles, rep cycles, coverage — and is not allowed to say anything about joint load or injury risk. Gemini 3.8 Flash interprets and explains, returns schema-constrained JSON, and states its own limitations, but never issues a safety verdict. ElevenLabs only speaks cues the local detector actually measured. The 3D replay, generated by Kimodo on an NVIDIA L4, is labeled a demonstration of the movement rather than a reconstruction of you, because one camera genuinely cannot measure where your joints are in space, and faking that would have been the easiest impressive-looking lie in the project. Even the optional Solana receipt only publishes a SHA-256 digest plus the exercise, rep count and duration — a gym can confirm you trained without ever receiving a frame of footage.

The only percentage left anywhere in the interface is "body tracked in X% of frames," which describes the camera, not the athlete.

It sounds like a limitation. I think it's the product. There are a lot of apps confidently grading bodies right now, and being the one that's honest about what a camera can see felt like the more useful thing to be.

Challenges

The funniest failure was early. I fed it a clean 14.4-second five-rep squat clip and it counted one rep. Landmark coverage was 100%, so the model had seen the body in every single frame — the bug was in my head, not the pipeline.

I'd written fixed thresholds. Go below this knee angle, hold it for this long, and you've earned a rep. Those numbers quietly described one specific body filmed at one specific angle, and everybody else gets told they didn't do the thing they just did. The fix was to stop having an opinion about how a squat should look and measure each person against their own standing position instead: take the 90th percentile of their knee angle as their upright baseline, enter a rep 14° below it, come back up within 8° of it. Depth and tempo stopped being gates and became things I report. Same clip, same landmarks, five reps — bottoms at 1.40, 4.20, 6.87, 9.67 and 12.67 seconds, within 0.002 s of the labels.

The second one cost me most of a day. The 3D replay was taking close to four minutes to generate. I stood up a persistent text-encoder service, got it to about 120 seconds, and concluded I needed a bigger GPU. Out of stubbornness I timed the generation step on its own first: three seconds. Everything else was overhead. A stray kimodo folder in my project was shadowing the installed Python package and forcing every job down a path that reloaded the full motion checkpoint from network storage — about 120 seconds — even when the model was already sitting warm in memory. Deleting a path took it to 3.1 seconds uncached and 0.04 cached, and it ran twelve sequential jobs without a queue failure. I have never been so annoyed to be so happy.

There was also a day I stopped feature work entirely because the app was running two visual languages at once. 31 rendered font sizes became 10, 40 spacing values became a 4px grid, 14 corner radii became 5. Nobody will ever notice that work, which is sort of the point of it.

What I learned

Bigger models aren't better models. I benchmarked all three pose variants at 15 FPS on labeled footage before picking one:

Variant	Mean inference	Coverage	Mean rep-bottom error
Lite (chosen)	23.05 ms	100%	0.002 s
Full	29.56 ms	100%	0.027 s
Heavy	85.67 ms	83.7%	0.015 s

The heaviest model was 3.7× slower and lost the body in one frame out of six. If I'd trusted my instinct I'd have shipped the worst one.

Measure before you spend money — I almost rented hardware to fix a folder name. Absolute thresholds smuggle in assumptions about who your user is. And the hardest calls were all subtraction: deleting the score ring and collapsing the design system took longer than most features and are the changes I'd defend hardest.

For the record, what's actually built and checked: five-exercise recognition and measurement, live spoken coaching, AI review with stated limitations, 3D replay on live GPU hardware, a SQLite participation ledger with a 200-point daily cap enforced inside a transaction, optional Auth0 login with the app fully usable signed out, a finalized public devnet transaction, 43+ unit tests, a clean production build, 9–11 Chrome end-to-end workflows, a WCAG A/AA scan with no violations, and no horizontal overflow at 390 px.

What's next

Validate against many more bodies, camera angles and lifts, because right now I've proven it works, not that it works for everyone. Move pose inference off the main thread. Then the honest version of the gym integration — authenticated analysis, a durable store, rotating-QR check-in — so participation becomes something a gym can verify instead of something they take my word for.

The limits I'm carrying forward, stated plainly: the movement score is an uncalibrated range-and-tempo rubric, single-camera 3D reconstruction is out of scope, and production anti-cheat isn't built. All three are named inside the app itself.

Which is the same idea as everything else here. Don't ask people to take your word for it. Show them the frame.

Built With

• css
• python
• typescript