Inspiration

Counting repetitions tells you how much you moved, but it does not explain the movement. For someone learning an exercise, questions like “Which muscles does this target?” and “What should I adjust?” can be harder to answer.

RepAnatomy grew from the idea of bringing movement feedback and anatomy education into one experience. We wanted users to see their exercise, explore the muscles involved, and understand the feedback without switching between workout videos, anatomy diagrams, and tracking apps.

What it does

RepAnatomy combines a camera-based exercise workspace with an interactive 3D anatomy atlas.

Users can practice five supported exercises: squats, push-ups, biceps curls, shoulder presses, and forward lunges. With a webcam or uploaded video, the application tracks observed repetitions, displays joint angles and movement ranges, and provides cues based on visible movement.

Alongside the exercise view, users can explore the expected primary and supporting muscles in 3D. These highlights teach anatomical roles; they do not claim to measure muscle activation.

The experience also includes:

Automatic exercise selection through repeated movement-pattern observations, with manual selection available.
Per-repetition duration and joint-range summaries.
Optional conversational coaching and spoken feedback.
Device-local daily workout summaries.
Reviewable weekly practice and meal drafts based on session evidence and user preferences. When lighting, framing, or tracking quality is insufficient, the system asks the user to adjust the capture and pauses counting. ## How we built it We built the interface with React, TypeScript, and Vite, and used Three.js for the interactive anatomy atlas. A MediaPipe Pose Landmarker runs inside a browser worker to estimate body landmarks while keeping inference separate from interface rendering.

A Python FastAPI backend receives sampled frames and landmarks. OpenCV 5 performs image decoding, resizing, grayscale conversion, sharpness and exposure analysis, optical-flow tracking, and coordinate transformations for joint geometry.

Those observations feed a deterministic coaching controller. Reliable movement can advance a repetition through its phases; poor evidence triggers reframing, lighting guidance, or tracking reacquisition. Separate movement thresholds and timing checks help prevent small fluctuations from becoming extra repetitions.

Optional Groq-powered language-model coaching explains structured session observations. It receives angle samples and rep summaries, rather than raw images, and cannot modify measured counts or override a session stop.

We also prepared a Docker build and AWS deployment infrastructure using ECR, ECS Fargate, an HTTPS Application Load Balancer, and CloudWatch. Cloud deployment and runtime validation remain pending.

Challenges we ran into

One challenge was deciding when the system had enough evidence to give feedback. A detected skeleton alone does not guarantee a usable observation. Blur, poor lighting, cropped joints, and unsuitable camera angles required explicit quality checks and recovery behavior.

Another challenge was repetition counting. Joint estimates fluctuate, so a single threshold can count the same movement repeatedly. We used separate thresholds for entering and leaving movement phases, timing constraints, and continuity checks to make counting more stable.

Running pose estimation beside a detailed 3D atlas also created competing demands on browser resources. We moved inference into a worker and reduced the embedded atlas’s rendering cost.

Finally, we needed to keep conversational coaching grounded in measurements. Separating the language model from counting and stop controls made its role clearer and its failures easier to handle.

What we learned

We learned that useful computer vision depends as much on handling uncertainty as on detecting movement. Asking for a clearer view can be more helpful than producing an unsupported answer.

We also learned to distinguish anatomical knowledge from sensor measurements, and software verification from real-world validation. Synthetic movement fixtures and browser tests helped us verify counting logic, quality gates, controls, and exports. They do not establish exercise accuracy across people or clinical effectiveness.

What's next for RepAnatomy

Our next priorities are deploying and validating the AWS build, evaluating consented exercise videos across varied people and recording conditions, and working with qualified trainers to assess repetition counts and movement cues.

The goal is to make exercise feedback easier to understand by connecting what users see themselves doing with the anatomy behind it.