FormCoach

Landing Page
Live Set Analysis
AI Personal Trainer
Video Upload Set Analysis

Inspiration

Recording a workout is easy. Knowing what to look for afterward is harder.

We wanted to make that review more useful, especially for people training without a coach. FormCoach started with a simple question: can an ordinary camera help someone understand their movement and approach their next set with more intention?

What it does

FormCoach turns workout videos into an interactive review. Users can upload a set, see an estimated skeleton over their movement, review detected reps, and jump back to specific moments in the video.

AI feedback organizes the takeaways into what went well and what to try differently. Users can also chat with an AI personal trainer about their set and save a summary to their device.

Our demo supports uploaded push-ups, lat pulldowns, incline dumbbell bench presses, and cable lateral raises, plus live push-up tracking.

How we built it

We built the frontend with Next.js, React, and TypeScript, and the backend with Python and FastAPI.

MediaPipe estimates body landmarks from video. Our exercise-specific analysis uses those landmarks to measure joint angles, identify movement phases, and count completed repetitions. Live and uploaded sessions share the same analysis foundation.

OpenAI powers conversational coaching and optional visual review of sampled video frames. We keep AI interpretations separate from measured results so generated feedback does not overwrite rep counts or measurements.

The interface brings everything together through synchronized playback, clear takeaways, and a locally saved workout log.

Challenges we ran into

Real gym footage is messy. Camera angles change, equipment hides joints, and other people enter the frame. We had to handle uncertain tracking instead of treating every pose estimate as reliable.

Counting reps also proved harder than simply checking an angle. Setup movements, pauses, and incomplete repetitions required careful handling and repeated testing.

Another challenge was making detailed feedback understandable. We iterated on the review screen to preserve useful context while removing repetition and keeping technical details out of the main flow.

Accomplishments that we're proud of

We connected video upload, pose tracking, rep analysis, AI feedback, playback navigation, and saved summaries into one working demo.

We're especially proud of moving beyond a generic chatbot: users can connect feedback to moments in their own workout and ask follow-up questions about that set.

What we learned

Working with real footage taught us to distinguish between what we can measure, what AI can interpret, and what remains uncertain.

We also learned that useful coaching depends on presentation. A few clear observations connected to playback can be more helpful than a screen full of technical measurements.

What's next for FormCoach

We want to improve reliability across camera angles and body types, expand exercise support, and make recording and reviewing sets easier on a phone.

We also want to evaluate the feedback with trainers and a broader collection of videos. Our next priority is making the existing experience more dependable before adding more features.