Cue

Cue - the PE teacher you never had

Most people have never had anyone check their form. Personal trainers are expensive, and a workout video on your screen can't see what you're doing wrong. Cue fixes that: point your phone camera at yourself, do a set of squats, and it coaches you like a real trainer in real time, in the browser, with no app or wearable.

What it does

Cue tracks your body with in-browser pose estimation, drawing a live skeleton over your camera feed. When you record a set, it analyzes every rep against real biomechanics; squat depth, knee tracking, forward lean, heel position, and left-right symmetry, and scores each one. Crucially, it's honest: if you hip-hinge instead of squatting, or don't really move at all, Cue refuses to score it and tells you why, instead of faking a number.

Then the part that makes it a coach, not a scorer: you can talk to it. After a set, an AI coach powered by GPT-5.6 reasons over your results. Ask "why couldn't I hit depth?" and it diagnoses the likely cause, like ankle mobility, and gives you a drill. Ask "how did my form hold up across the set?" and it reads the pattern rep by rep, telling you whether you broke down from fatigue or technique. It adapts to injuries, time limits, and goals you mention.

How we built it

Cue has two layers working together. In-browser pose estimation (MediaPipe) detects 33 body landmarks and feeds a deterministic scoring engine, we wrote all the geometry is computed in code and normalized against body scale, so it works consistently across different phones and camera distances. That structured analysis is then handed to GPT-5.6, which does the reasoning and coaching in natural language.

The most important decision we made was not to let the language model do everything. Our first version had the model score the reps, and it was unreliable, good and bad reps scored the same, and the advice was generic. So we split it: deterministic code measures and scores, GPT-5.6 reasons and coaches. That's what makes Cue both reliable and genuinely intelligent. We also caught it scoring a hip-hinge as a good squat (depth was the only failing check), so we added a knee-flexion gate that makes it honestly reject non-squats.

We built the entire thing, the Next.js app, the pose pipeline, the scoring engine, and the coaching chat, through Codex in a single continuous session, and deployed it on Vercel.

Try it

Live demo: cue-fawn.vercel.app, open /coach on a phone or laptop with a camera, allow access, stand side-on with your full body in frame, and record a set. No login required.

What's next

The squat is our proof of concept. The same architecture , deterministic detection feeding an LLM coach, extends directly to deadlifts, push-ups, planks, and running gait. We went deep on one movement to prove the coach genuinely understands what it's watching.

Built with: Next.js, TypeScript, Tailwind, MediaPipe, OpenAI GPT-5.6, Codex, Vercel.

Built With

• codex
• mediapipe
• next.js
• openai
• tailwind
• typescript
• vercel