Spottr

Homepage
Overall Progress
Workout List
Starting Workout
Beginning Squat
Mid Squat
Workout Review
Detailed Workout Report
Skill Tree Grid
Skill Tree Linear
Asking Coach for Workout Plan
Coach's Recommended Workout Plan

Inspiration

Personal trainers are very expensive, especially for students. However, getting expert feedback on your workouts is the best way to improve your consistency, accuracy and healthspan. To more this more accessible, we introduce Spottr. Spottr is a gym coach that never gets tired: one that watches your form, counts for you, and pushes you with a big, motivating voice.

What it does

We use your phone camera to watch you perform free-weight exercises, linking with biological signals we extract from your face and your smart watch. We use computer vision algorithms to evaluate your technique and use an AI coach to give you audible feedback so it feels like you are in the room with an actual personal trainer. We have support for 19 exercises with lessons, goals, skills and reports. We trained an AI couch to sound just like Arnold - the king of bodybuilding. Arnold sees you, talks to you, plans with you and learns you.

How we built it

Built native iPhone and Apple Watch apps with SwiftUI, sharing workout controls, sensor data and session history.
Used MediaPipe’s pretrained pose model to locate body landmarks, then built angle-based logic to assess push-ups and squats and count complete repetitions.
Connected a Raspberry Pi side camera to the app, alongside the phone’s front-camera preview.
Built a coaching engine that combines form, rep timing, completed sets and available heart-rate data to decide when to encourage, slow down or stop.
Integrated Gemini for workout planning and reports, with structured exercise selections that users review and approve.
Used ElevenLabs to create motivating coach audio, caching short cues for fast playback during the demo.
Built integrations for Tiger Data’s time-series storage, Snowflake analytics and Presage camera-based vitals, with a backend prepared for Vultr hosting.
Added a branching skill tree, streaks, exercise lessons and progress reports to make improvement visible and rewarding.
Developed in parallel through GitHub branches and pull requests, testing with synthetic workouts and the existing iPhone and Watch simulators.

Challenges we ran into

Shape exact voice style we wanted into ElevenLabs. Advanced models weren't able to be used because we were using prerecorded audios and could not say exactly they said.
Getting AI voice audio generation synced with movements was tricky because we had to prioritise speed and accuracy.
Arduino did not have a USB port so we needed to use a Raspberry Pi. Camera we had was using too much power so we had use another one but it had its own corrupted video feed.
Getting the computer vision models working to accurately assess a complete pushup or squat.
Syncing Apple Watch with the iPhone app directly to work seamlessly.

Accomplishments that we're proud of

Training the ElevenLabs to sound like David Goggins and Arnold Shwarzenegger.
Training the computer vision models to track body pose and reps for pushups and squats.
Developing a full iPhone and Apple Watch companion app.
Finding a way to integrate the different side tracks together in a way which makes sense.
Adding a skill tree which gamifies your own exercise stats and skills.

What we learned

Developing and training full computer vision pipelines.
Learning all the new SDKs.
Working with all team members, using AI, working extremely rapidly but in agile way.