DriftGuard

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for DriftGuard

Inspiration

Most AI fitness apps grade every rep against one fixed idea of "perfect form." That template usually comes from somebody else's body, somebody else's mobility, somebody else's camera angle. But that's not really how people fail a set. Athletes don't drift away from a stranger's ideal squat. They drift away from their own starting point, the one they had on rep one. We wanted to measure that instead. Not "is this technically a good squat," but "has this specific person's movement started changing from where they began."

What it does

DriftGuard watches you squat through a webcam, or a video you upload, and learns your baseline from your first four controlled reps. Depth, trunk lean, hip range, tempo, all of it gets recorded. Every rep after that gets scored 0 to 100 based on how far it's drifted from your own baseline, and you get a breakdown of exactly which part of your movement is causing the score to move. At the end of a set you see the full trend and the exact rep where things started to change. And here's the part we like most: everything runs right there in your browser. No video ever gets uploaded anywhere.

How we built it

We used MediaPipe's Pose Landmarker Lite for pose estimation, 33 body landmarks per frame, running entirely on-device. The wasm files and the model itself are bundled locally too, so once the page loads there's zero network dependency.

From there it's all our own code. We smooth the landmarks with an exponential moving average so the skeleton doesn't jitter around. We built a state machine with hysteresis to break a continuous video into individual reps, standing, descending, bottom, ascending, tuned carefully so one noisy frame can't invent a rep that didn't happen or miscount a half-movement. For each rep we pull out the features that matter: minimum knee angle for depth, max trunk lean, normalized hip range, how long the descent and ascent took.

The baseline comes from the athlete's first four reps, using the median and something called MAD (median absolute deviation), with a floor built in so an unusually consistent calibration set doesn't make the whole thing oversensitive. Then we score drift as a weighted, clamped z-score against that baseline, and we show the contributor breakdown right in the UI. Nothing about the score is a black box.

The whole thing is Next.js 16, TypeScript, Tailwind, Recharts, deployed on Vercel. And we wrote real Vitest tests for the deterministic core, the angle math, the rep segmentation, the calibration, the scoring, completely separate from the camera and UI layer.

What makes it different

Pose-based form coaches are everywhere at hackathons now, honestly. MediaPipe already solves "find the body in the frame," so that's not really our contribution. Ours is what happens after the landmarks show up. Personal calibration instead of one universal threshold. Drift measured across a whole set instead of judging one rep in isolation. And a score you can actually see the reasoning behind, instead of a number that just appears.

Challenges we ran into

Getting the rep counter right was harder than it sounds. We had to reject standing jitter and half-reps that never hit real depth, without missing actual reps. Confirmation-frame hysteresis ended up being the fix, way more reliable than a single threshold.

Then Windows threw us a curveball. The Vercel CLI kept failing with a symlink permission error during local builds, so we switched to importing straight from GitHub through Vercel's dashboard instead, which builds remotely and just skips the problem entirely.

We also caught real mobile bugs during testing, a header badge wrapping mid-word, a stat card breaking awkwardly on a narrow screen, stuff you only catch by actually testing at phone size instead of assuming desktop is good enough.

And honestly, the biggest one: we didn't trust it until we ran it against real human squat footage, not just unit tests. Synthetic data can lie to you.

Accomplishments that we're proud of

We got the whole pipeline, tracking, rep counting, calibration, scoring, verified end to end against actual human movement, not just test cases we wrote ourselves. Ran a three minute session, 22 reps, zero console errors. It's fully responsive and we actually tested that on real phone-sized viewports. And there's no backend, no external API, no LLM anywhere in the core. Just deterministic code sitting on top of an on-device pose model.

What we learned

Personal baselines beat universal thresholds for something like this, but only if you pair them with confidence gating. The system needs to be willing to say "I can't see you clearly enough to score this" instead of guessing. And real-time computer vision takes almost as much engineering after the model gives you output as it does before. The landmarks are just the starting point.

What's next for DriftGuard

We want to add a "baseline ghost," your current pose shown next to a faint outline of your calibrated baseline, so drift is something you can literally see, not just read as a number. Beyond that, more movements than just the squat, and some kind of session history so people can track consistency over time instead of just within one set.

Built With

• mediapipe
• next.js
• recharts
• tailwind-css
• typescript
• vercel
• vitest