spotter

A little guidance, every set.

Comment

Inspiration

Neither Thanmay nor I came into this as gym experts. We’re grad students who wanted to get into the gym, but walking in for the first few times can be pretty intimidating. There are machines everywhere, half of them have names you’ve never heard of, and you’re left wondering if you’re even using the right one.

You can find a workout plan online in five minutes. Figuring out where a machine is, how to use it, and what to do when it’s already taken? That’s a different story.

We wanted to make that experience less overwhelming. What if you had a voice companion that knew your gym’s layout, guided you to the right equipment, walked you through each set, and helped you adjust when things didn’t go as planned?

That’s how Spotter came to be. We built Mat to help beginners navigate the gym and stick to a workout they’ve actually approved, without constantly checking their phones or feeling lost between machines.

We’re still figuring out the gym ourselves. We just wanted to build something that would’ve made those first few visits a whole lot easier.

What it does

Spotter starts with your workout, not a generic AI conversation. Sign in with Google, pick the muscle groups you want to train on the body map, and choose from our catalog of 18 gym stations. You can edit your sets, reps, and weights before approving the plan.

Once you start, Mat takes over as your voice companion. It knows the gym layout and helps you navigate between machines, guides you through your sets and rest periods, and adapts when equipment is taken. You can use voice or on-screen buttons to tell it what happened, and every completed set gets saved with its time and weight.

We also built a weekly planner, so you can set up your workouts from Monday through Sunday without overwriting a session you’re doing today. You can customize Mat’s name and avatar, give it a reason you train, and choose English or Spanish.

And because we wanted the experience to extend beyond a phone screen, we added a browser-based watch control panel and a SwiftUI watch app that can join the same workout session.

How we built it

We built the phone and web experience using Expo and React Native, with a FastAPI backend, Pydantic schemas, and SQLite handling the workout engine and persistent state. The gym catalog, workout events, and voice tools use versioned contracts so the app, voice agent, and watch all speak the same language.

One of our biggest design decisions was keeping Mat separate from the actual workout state. Mat can interpret what you say, but it cannot directly mark a set as complete. Voice commands and button presses go through the same backend actions, which validate the current exercise, set, and workout phase before making a change.

We also made the session resilient to duplicate taps and stale requests. Rest deadlines are saved in the database, so refreshing the app or restarting the server doesn't automatically end a rest or mark a set as finished.

For the muscle selection screen, we built a rotating anatomy model for the web, with a simpler diagram and labeled buttons as a fallback when WebGL isn't available. We added fatigue warnings for plans with too many sets targeting the same muscle, and integrated ElevenLabs for voice while keeping provider keys on the server.

Challenges we ran into

The biggest challenge was getting a browser, a voice agent, and a watch to agree on what was happening in one workout.

Imagine saying that you've finished a set, tapping the button at almost the same time, and then having the voice request retry. Without the right safeguards, one set could become two. We made the backend the single source of truth and gave each action an ID so duplicate requests couldn't count twice.

Working across Windows and Mac machines added another challenge. We couldn't just build our parts independently and hope they'd fit together. The schemas, event names, examples, and tests had to stay aligned, especially as the watch and web experience came together.

Voice was tricky in a different way. Microphones fail, speech recognition gets things wrong, and an AI saying something confidently doesn't mean it actually happened. We kept buttons as a fallback and made sure Mat only confirms a state change after the backend accepts it.

Accomplishments that we're proud of

We're proud that Spotter goes beyond generating a workout plan. A user can choose their muscles, approve a workout, navigate the gym, work through sets and rest periods, and return to a session with their progress intact.

The weekly planner is separate from today's workout, so planning ahead doesn't accidentally overwrite what you're doing right now. We also got the same action model working across the main app, a browser-based watch panel, and the SwiftUI watch join flow.

Most importantly, we built safeguards around the parts that matter. Duplicate requests don't count a set twice, stale actions can't silently change the wrong exercise, and a server restart doesn't erase a saved rest deadline.

We deliberately left out features we couldn't honestly support, like camera-based form analysis, live equipment occupancy detection, and automatic rep counting. We'd rather have a smaller demo that works than a flashy one that pretends to do more than it can.

What we learned

One thing we learned pretty quickly is that a workout isn't just a conversation. It's a sequence of actions where timing and state matter. If Mat misunderstands a sentence, that shouldn't mean a set magically gets completed.

We also learned how important shared contracts are when multiple parts of a project are being built in parallel. Agreeing on what an action means, how the backend validates it, and what happens when it gets sent twice saved us from a lot of integration headaches.

And perhaps the biggest lesson was knowing what not to build. It's tempting to add more AI features to make a hackathon demo look impressive, but reliability matters more when someone is relying on your app in the middle of a workout.

What's next for Spotter

First, we want to get Spotter running as a hosted demo and bring the full experience to the iPhone, including native Google sign-in and a published ElevenLabs agent for live English and Spanish voice interactions.

We also want to finish the paired watch experience, so someone can control their workout from their wrist while Mat speaks through their phone, even when the connection drops and comes back.

Beyond that, we want to get the machine guidance professionally reviewed and make the transition from a saved weekly plan to that day's actual workout feel seamless.

The goal stays the same: make the gym a little less intimidating for people who are just getting started. We'll keep the equipment catalog curated and focus on making the core experience more reliable before adding anything more ambitious.

Built With

Share this project:

Updates

Submission history