FormFlow

FormFlow is a real-time exercise form analysis platform that watches you lift through your webcam, scores your movement quality rep by rep, and instantly broadcasts form failures to your workout group

Comment 1

Inspiration

Bad form is the silent epidemic of fitness culture. It's the #1 cause of gym injuries, and yet most people have no real-time feedback mechanism to catch it before it becomes a problem. Personal trainers are expensive and can't be everywhere. Mirrors only show what's in front of you. And your gym friends? They don't want to be the one to say something.

We also noticed something deeper: working out alone is boring and easy to quit. Accountability is the single biggest predictor of whether someone sticks to a fitness routine — but existing apps treat social features as an afterthought, a leaderboard bolted on after the fact.

We wanted to build something that fused real-time biomechanical intelligence with genuine social accountability: a platform where your camera becomes your coach, and your crew keeps you honest.

What It Does

FormFlow is a real-time exercise form analysis platform that acts as an always-on AI coaching assistant — built entirely in your browser, no app install, no wearable required.

Point your webcam at yourself and start lifting. FormFlow's pose tracking engine analyzes your movement frame by frame, scoring every single rep from 0 to 100 based on the smoothness and control of your motion. It detects specific biomechanical faults — knee valgus on squats, lumbar flexion on deadlifts, elbow flare on bench press — the moment they happen.

But the social layer is where FormFlow gets ruthless. When your form breaks down, a WIPEOUT_EVENT fires instantly over Socket.IO to every member of your workout group. No hiding sloppy reps. No cheating through a set when nobody's watching. Your whole crew sees it in real time.

After your session, a replay panel shows your full fluidity curve and kink timeline so you can see exactly which rep you started breaking down and why. A group leaderboard ranks everyone by their highest fluidity scores, and completing sessions earns hydration credits — a small but satisfying gamification hook that rewards showing up consistently.

How We Built It

Pose Analysis Pipeline (Browser). We built the entire movement analysis stack in JavaScript running locally in the browser. MediaPipe Pose streams skeletal landmarks frame by frame. angleUtils.js extracts the key joint angles — hip angle, knee angle, lumbar flexion — from those landmarks. A fluidity scorer applies an exponential moving average to the angular velocity of those joints, normalizing and inverting it into a 0–100 smoothness score. A rule-based kink detector evaluates biomechanical thresholds specific to each exercise type on every frame within an active rep. A phase state machine (IDLE → DESCENDING → BOTTOM → ASCENDING → TOP) handles rep counting automatically using joint-angle thresholds and a minimum rep duration guard — no button presses needed.

Backend (Node.js / Express / MongoDB). The Node.js backend handles persistence, real-time fan-out, and leaderboard aggregation. Workout sessions are stored as rich MongoDB documents containing the full poseSnapshots array — one entry per rep — alongside derived statistics (averageFluidity, totalKinks, maxFluidity) that are auto-computed via a Mongoose pre('save') hook, keeping derived data always consistent without any application-layer coordination.

The Reactive Core (MongoDB Change Streams). Rather than polling or building a separate message broker, we watch the sessions collection with a MongoDB Change Stream. The moment a session document with detected form faults lands in the database — from any backend instance — the change stream fires and Socket.IO broadcasts the WIPEOUT_EVENT to every member of that FriendPool group room instantly. MongoDB itself is the event source. This architecture scales horizontally without changing a line of application code.

Leaderboard (MongoDB Aggregation). The top-fluidity leaderboard is served entirely by a MongoDB aggregation pipeline: $group by userId capturing $max fluidity and $sum session count, $sort descending, $limit to top N, and $lookup to join display names — all in a single database round-trip. No application-layer sorting, no N+1 queries.

Frontend (React + Vite). A three-tab Progressive Web App — Home for onboarding, Gym for the live camera session experience, and Social for the leaderboard and friend feed — with a live fluidity gauge, rep counter, wipeout overlay, and post-session replay chart.

Challenges We Ran Into

The hardest technical challenge was making pose-based rep detection reliable across different body types and camera angles. MediaPipe landmarks are noisy, and naively thresholding joint angles produced false positives constantly — micro-movements being counted as reps, or good reps being flagged as kinks. We solved this by adding a minimum rep duration guard to the phase state machine and tuning the EMA smoothing factor to suppress high-frequency noise without introducing lag that would delay real fault detection.

The second major challenge was the real-time architecture. Our first instinct was to emit wipeout events directly from the REST handler that receives session data. This worked in single-instance development but would have broken immediately under horizontal scaling — two backend instances wouldn't share socket state. Switching to a MongoDB Change Stream as the event source solved this cleanly: any instance that saves a session triggers the change stream, which fans out through Socket.IO to the correct group room regardless of which server originally handled the write.

Finally, keeping the browser-side pose pipeline from tanking the UI frame rate required careful profiling. MediaPipe is GPU-accelerated but the angle computation and state machine logic running on every frame added up. We offloaded the heavier aggregation to snapshotBuffer.js, which batches per-frame data into per-rep summaries asynchronously rather than blocking the render loop.

Accomplishments That We're Proud Of

We're most proud of how much of the real-time architecture MongoDB enabled natively. Change Streams gave us reactive push semantics without Kafka, RabbitMQ, or any external message broker. The aggregation pipeline gave us leaderboard ranking without a separate analytics database. Mongoose pre('save') hooks kept derived stats consistent without any application-layer coordination logic.

Building a full biomechanical analysis pipeline — pose landmarks, joint angle extraction, EMA fluidity scoring, heuristic kink detection, phase-based rep counting — entirely in the browser with no native app, no wearable, and no server-side inference is something we're genuinely proud of. It means FormFlow works for anyone with a laptop and a webcam.

And the social mechanic genuinely works. When a wipeout alert fires mid-set and everyone in your friend pool sees your form break down in real time, you fix it. The accountability loop is tight enough that it actually changes behavior.

What We Learned

We learned that MongoDB is a dramatically more capable real-time infrastructure layer than most people use it for. Change Streams turn a document database into an event bus. Aggregation pipelines replace entire analytics microservices. Mongoose hooks enforce invariants at the data layer rather than scattering that logic across route handlers.

We also deepened our understanding of the tradeoffs in browser-based ML inference. MediaPipe is remarkably fast, but the architecture of the analysis pipeline — how you structure the frame loop, when you batch vs. stream, how you handle the lag between visual input and score output — matters enormously for the feel of the product. A technically correct fluidity score that arrives 500ms late feels worse than a slightly noisier score delivered in real time.

Most importantly: social mechanics are infrastructure, not features. Building the real-time fan-out as a first-class architectural concern — not an afterthought — is what makes the group accountability loop actually work.

What's Next for FormFlow

Rep-streaming API. Replace the bulk session upload with a streaming PATCH /api/sessions/:id/rep endpoint so the backend has per-rep fidelity in real time, enabling live coaching nudges mid-set rather than post-session analysis.

ML-based kink classification. Replace hardcoded biomechanical thresholds with a lightweight TensorFlow.js model trained on labeled lift footage. Heuristic thresholds work well for textbook form but fail on edge cases — atypical anatomy, unconventional but effective technique, and advanced lifters who intentionally break "rules."

Group challenges. Weekly FriendPool fluidity challenges with push notifications when a friend beats your score — turning the leaderboard from a static ranking into an ongoing competitive layer.

Richer session history. Store full joint angle arrays and fault flags per rep to power a detailed post-workout breakdown — a visual record of exactly how your mechanics degraded across a training session.

Mobile PWA. Add a service worker and web app manifest so FormFlow installs natively on iOS and Android. The entire pose analysis pipeline already runs in the browser — mobile is a packaging problem, not an engineering one.

Audio coaching. Real-time text-to-speech cues ("brace your core", "slow the descent") triggered by kink detection events, turning FormFlow from a passive scoring system into an active verbal coach.

Built With

Share this project:

Updates