Workout Coach

WorkoutCoach uses AI coaching personas, real-time computer vision rep tracking, and intelligent workout generation, all of which is grounded in your actual training data. Your lifting data, decoded.

Comment

Inspiration

WorkoutCoach

Your lifting data, decoded.

Inspiration

As a competitive badminton player and CS student, for years there wasn't really a tool to track my badminton progress. When I went to college, there was a tool to track workouts But... The problem? None of them understand training. They'll count your reps but can't tell you that your squat volume is trending down while your RPE is creeping up which is a textbook overtraining signal. I wanted a tool that could watch my form, know my numbers, and coach me with real data. WorkoutCoach is the AI-powered training platform I personally wish existed.

What It Does

WorkoutCoach combines three AI systems into one platform:

  1. Multi-Persona AI Chat Coach — Five specialized coaching personas (Strength, Hypertrophy, Powerlifting, Recovery, Movement) analyze your actual workout history and stream personalized coaching advice via Claude.

  2. AI Workout Generator — Describe your goals in plain language (e.g., "heavy bench day with accessories") and get a structured, periodized workout plan built from your real training data. Weights are matched to your history — not random defaults.

  3. Real-Time Computer Vision Form Tracking — YOLOv8-pose estimates 17 COCO keypoints from your webcam at 10 FPS. A custom rep counter computes joint angles and uses a state machine to count reps across four exercises: squat, bicep curl, overhead press, and lateral raise.

How It's Built

Frontend: Next.js 16 / React 19 / TypeScript / Tailwind CSS 4 / Framer Motion. The landing page features orchestrated stagger animations, a typewriter command preview, and spotlight cursor effects. Chat UI streams token-by-token via ReactMarkdown.

AI Backend: Next.js API routes connect to the Anthropic Claude API. Chat uses Claude Haiku 4.5 for low-latency streaming via SSE; workout generation uses Claude Sonnet 4 for reliable structured JSON output. Each request injects the user's recent workout history (up to 36 entries) and profile data into persona-specific system prompts.

Computer Vision: A FastAPI sidecar runs YOLOv8-pose inference. The browser captures webcam frames as JPEG blobs and sends them over WebSocket. The server returns keypoints, which the client renders as a skeleton overlay on HTML5 Canvas. The rep counter uses:

  • 5-frame rolling average for angle smoothing
  • Exercise-specific phase thresholds (e.g., squat
  • Back-pressure flow control (waits for server response before sending next frame)

Data Layer: Supabase (Postgres) stores all workout data. CSV import auto-detects Strong app format vs. generic CSV via PapaParse. The inference server is containerized with Docker.

Challenges

  • WebSocket reliability — Maintaining a stable connection between the browser and inference sidecar required careful reconnection logic, especially when switching exercises mid-session.
  • Rep counting accuracy — Naive angle thresholds produced false positives. Adding a 5-frame smoothing buffer and minimum confidence gating dramatically reduced noise.
  • Multi-model orchestration — Choosing the right Claude model per task (Haiku for speed, Sonnet for structured output) required benchmarking latency vs. output quality tradeoffs.
  • Context window management — Injecting workout history into every API call without exceeding token limits meant carefully capping at 36 entries for chat and 15 for generation.
  • Deployment — I've had trouble finding a suitable service to host to backend on remotely without paying extra money so currenty the application runs in the web except for the video coach part. In order to run that part you will have to clone and run it locally.

What I Learned

Building WorkoutCoach reinforced that the best AI products aren't about bolting an LLM onto a CRUD app , it's about deeply integrating AI into the data flow. The multi-persona system prompt design was a lesson in how much coaching quality depends on prompt architecture, not just model capability. And combining computer vision with LLM coaching (video form data → chat feedback) showed the power of multi-modal AI pipelines even without a single multi-modal model.

Built With

next.js · react · typescript · tailwind-css · anthropic-claude · yolov8 · fastapi · supabase · docker · framer-motion · websockets · opencv

What's next

Update this app to support accounts and everyone can have their personal data uploaded. Right now my data is uploaded as a sample just for demo purposes.

Update deployment as well to get the inference server running in the cloud.

This app might be a start on the real time badminton coach app. Stay tuned.

Built With

Share this project:

Updates