SpotOn

History/Movement Library Page- Dark Mode
Home Page- Dark mode
Home Page-Light Mode

Inspiration

Squat feedback is often difficult to access without a trainer, specialized equipment, or sharing personal videos online. We wanted to build a simple, private tool that helps people understand their squat movement using just a phone or laptop camera.

We realized that while fitness apps are great at counting reps and tracking workouts, they don't always help people understand how they're moving. We wanted to change that by making movement analysis more accessible, without requiring expensive equipment, wearable sensors, or a gym membership.

That's how SpotOn started: a way to make squat feedback more accessible, understandable, and privacy-conscious.

What it does

SpotOn is a camera-based squat analysis tool that helps users observe, review, and compare their squat movements.

Users position their camera to capture a side view of their body, complete a short calibration, and perform a set of bodyweight squats. Using computer vision, SpotOn tracks body landmarks and analyzes movement patterns without requiring wearable devices.

The application allows users to:

Track squat movements: Detect body landmarks and identify repetitions using pose estimation.
Review their form: Replay recorded movements with a skeleton overlay and joint-angle measurements.
Understand their performance: View measurements and movement observations from each set.
Compare workouts: Compare two compatible squat sessions to see differences in movement patterns, depth, and timing.
Get AI-powered explanations: Optionally use Gemini to translate technical measurements into easier-to-understand feedback.
Receive voice guidance: Use optional ElevenLabs-generated narration for camera setup, calibration, and repetition announcements.
Keep their data private: Store movement landmarks locally without uploading raw workout videos to a server.

SpotOn also includes workout history and replay controls, allowing users to revisit their sessions instead of relying on feedback they might forget immediately after exercising.

Our goal is to make movement analysis feel less intimidating and more useful for everyday people.

How we built it

We built SpotOn as a web application using Next.js, React, TypeScript, and Tailwind CSS, with a focus on creating an accessible experience across desktop and mobile-sized screens.

For the core movement analysis, we integrated Google MediaPipe Pose Landmarker, which identifies 33 body landmarks from camera frames. We use these landmarks to calculate joint angles, track movement phases, and analyze squat repetitions.

One of our main technical decisions was to process pose data directly in the browser. We used browser-based inference and Web Workers to separate pose estimation from the main interface where possible, helping keep the application responsive during camera processing.

We also developed a system to record movement landmarks rather than full videos. This allows users to replay their squats using a reconstructed skeleton, inspect joint-angle traces, and compare sessions while reducing the amount of sensitive information stored.

For session comparisons, we built logic to align important movement phases, such as the start, bottom, and end of a repetition, while preserving the actual timing of each movement.

We integrated Google Gemini as an optional explanation layer, allowing users to better understand the measurements produced by our analysis system. Instead of sending video footage to the AI model, the application uses limited, aggregated movement measurements.

We also incorporated optional ElevenLabs-generated voice guidance to make the workout experience more interactive.

For storage, we used IndexedDB to save session data locally in the browser. We implemented offline-support features through a Progressive Web App architecture.

Finally, we used Vitest and Playwright for automated testing, including simulated movement sequences and browser-based checks.

Challenges we ran into

One of our biggest challenges was figuring out how to turn raw pose-estimation data into feedback that someone could actually understand.

MediaPipe provides coordinates for body landmarks, but those coordinates alone don't tell someone whether their squat movement is consistent or how one repetition compares with another. We had to build additional logic to calculate joint angles, identify movement phases, and handle missing or unreliable landmark detections.

Camera positioning was another challenge. The quality of pose tracking depends on factors such as lighting, visibility, and camera angle. We designed a side-view setup and calibration process to make recordings more consistent, but real-device accuracy and performance still need further testing.

We also had to think carefully about privacy. Uploading workout videos to a server would have made some parts of the application easier to implement, but it went against one of our main goals. Instead, we worked on keeping the core analysis and saved movement data within the user's browser.

Another challenge was making meaningful comparisons between workouts. Simply comparing two sets frame by frame wouldn't work because people perform squats at different speeds. We addressed this by aligning key phases of each repetition while retaining their original durations.

Finally, integrating computer vision, local storage, AI explanations, and voice guidance into one cohesive application required us to think carefully about performance, error handling, and the overall user experience.

Accomplishments that we're proud of

We're especially proud of building something that goes beyond simply counting repetitions.

Our biggest accomplishment was creating an end-to-end movement analysis workflow, from camera setup and calibration to squat tracking, local storage, skeleton replay, and session comparison.

We're also proud of:

Privacy-first design: Core movement processing happens locally, and users don't need to upload raw workout videos.
Skeleton-based replay: Users can revisit their movements and inspect joint angles without storing full video recordings.
Meaningful workout comparisons: We developed a way to compare repetitions based on movement phases rather than relying only on timestamps.
Explainable AI integration: Gemini provides an optional way to interpret measured movement data rather than replacing the underlying analysis.
Accessible interaction: Voice guidance and a browser-based interface help make the tool easier to use without specialized equipment.

What we're most excited about is that SpotOn brings together computer vision, movement analysis, and thoughtful user experience design to address a problem that many people encounter when exercising alone.

What we learned

Working on SpotOn taught us that detecting movement and understanding movement are two very different problems.

We learned how browser-based computer vision works, how to process body landmarks using MediaPipe, and how geometric calculations can turn coordinates into meaningful measurements.

We also gained experience working with Web Workers, local browser storage, automated testing, and integrating AI services into a larger application.

One important lesson was that AI-generated explanations are only as useful as the information behind them. We learned to separate deterministic measurements from AI interpretations so that the underlying results remain available even when an AI response isn't.

We also learned how important it is to design around real-world limitations. People have different devices, camera setups, and movement patterns, so building a useful fitness tool requires more than just making the model run.

Most importantly, we learned to balance technical ambition with usability and privacy. A tool can have impressive technology behind it, but it only matters if people can understand and trust what it's telling them.

What's next for SpotOn

We see SpotOn as a starting point for making personalized movement feedback more accessible.

Our immediate next step is testing the application with real users and physical devices to evaluate camera performance, repetition detection, and the consistency of our movement measurements.

From there, we'd love to expand SpotOn with:

More exercises: Support movements such as lunges, push-ups, and deadlifts.
Real-time feedback: Provide more immediate movement cues during a workout.
Personalized progress tracking: Help users understand how their movement patterns change over time.
Improved accessibility: Refine voice guidance and mobile usability.
Better movement analysis: Improve calibration, detection reliability, and support for different camera angles and body proportions.

In the long run, we want SpotOn to become a practical movement companion for people who want to exercise with more awareness and confidence.

Our vision is simple: you shouldn't need an expensive setup or a personal trainer just to start understanding how your body moves.