Swole Patrol

Inspiration

We’ve all seen sketchy reps that lead to aches and plateaus. Since ~55% of gym injuries are linked to poor technique, we wanted a browser-based spotter that fixes form in real time: no wearables, no app install.

What it does

Swole Patrol watches your workout with a webcam, tracks your body with pose estimation, and:

Gives instant, on-screen tips (and optional voice cues)
Counts reps / plank hold time and scores form
Logs notable issues for your session history
Generates a short set summary with three personalized tips

How we built it

PoseDetector (MediaPipe): streams webcam frames → body landmarks
FormAnalyzer: exercise-specific rules (angles, alignment, tempo) for Plank and Bicep Curl; computes rep/hold counts, form score, and feedback severity
Workout UI: canvas overlay draws the skeleton; panels show status, tips, and history
AI Coach: sends the last reps/issues to Gemini and returns a concise summary + 3 actionable tips; optional TTS for voice coaching
Frontend: vanilla JS/HTML/CSS with a time-of-day theme and smooth UI details

Challenges we ran into

Noisy landmarks: needed smoothing + stability thresholds to avoid flickery feedback
Camera angles: users must face sideways for planks—added detection and guidance
Over-talkative coach: throttled voice cues and deduped feedback logs
Robust AI parsing: enforced strict JSON responses and fallbacks when the model drifts

Accomplishments that we're proud of

Real-time coaching that actually feels like a spotter, entirely in the browser
Stable rep/hold tracking with actionable, non-spammy tips
Set-level AI summaries that highlight patterns (not just one-off errors)
A clean UX with history, best plank time, and friendly visuals

What we learned

Small UX details (debouncing feedback, grouping issues) make AI coaching feel human
Clear camera guidance beats fancy models when it comes to reliability
Mixing deterministic rules with LLM summaries gives the best of both worlds: trustworthy corrections + motivational coaching