Spotter

Your phone is the camera. Your Fire TV is the coach.

Comment

The problem

Working out at home, nobody watches your form. Half squats feel like full ones, you lose count, you rush, and you have no idea whether you're improving. A personal trainer fixes all of that, but costs more than most people will pay, and fitness apps on a phone are too small to watch from across the room mid-squat.

Who it's for

People who train at home in front of their TV: beginners who want to know a rep "counts", busy people who want a 10-minute set with real feedback, and families who share one TV (up to 4 players, each with their own progress).

What Spotter does

Spotter turns a Fire TV into a coach that watches your form.

  1. Open Spotter on Fire TV. Scan the QR code with any phone. There's no app to install and no account.
  2. Lean the phone against something across the room. It tracks your body on the phone and sends only body points, never video, to the TV.
  3. The Fire TV app counts every rep live for squats, lunges and jumping jacks. A rep only counts at full depth; a half rep gets "Go deeper" on screen and out loud.
  4. It trains you like a coach: 3 sets of 12 squats, a rest countdown between sets ("Great set! Take a breather.", "Set two. Let's go!"), and Skip rest when you're ready. During each rest, the coach tells you the one thing to fix next, from the set you just did ("2 reps didn't reach full depth. Next set: every rep all the way down."), an AI rewrites it in a coaching voice and Polly says it by name; the next set checks it, and "Focus kept" is celebrated.
  5. It talks to you by name ("Let's go, Ramy!", "Last one, Ramy!") with an Amazon Polly voice, and celebrates clean streaks and your deepest rep with funny cartoon stickers.
  6. After the workout, the coach sums it up: pace, depth, clean reps, and a target for next time. Finish every set with clean form and the next workout asks for 2 more reps per set.

How it uses Fire TV (Vega OS)

The whole TV experience is a Vega OS app written in React Native for Vega:

  • remote-first design: D-pad focus on every screen, a 1280×720 layout scaled to any TV, and focus shown by border and fill so it's readable from 3 m;
  • Vega AsyncStorage keeps players, names and progress on the TV, so there's no account;
  • Vega W3C Media (AudioPlayer) plays the coach's voice on the TV;
  • the Vega CLI and Virtual Device built, tested and packaged the app (release .vpkg), which is live on the Amazon Appstore for Vega Fire TVs (version 1.1.0 with the between-set coach, published 2026-10-08; each release passed primary, content policy and functionality validation).

How it uses AWS

  • API Gateway WebSocket + Lambda + DynamoDB: a serverless relay that pairs the phone and the TV with a 6-character code and carries about 15 pose frames per second, 88 ms median phone-to-TV.
  • Amazon Polly: 113 coach phrases, plus each player's name, made once and cached, and the between-set coach's line, voiced live during the rest.
  • AWS CDK: the whole stack as code.

How it's built, briefly

On-device pose tracking (MediaPipe) on the phone → the WebSocket relay on AWS → the Vega app, which runs the rep counting, cues, rewards, voice and coaching. The AI coach rewrites the summary and the between-set line in friendlier words but is never allowed to invent a number, and never sees the player's name. The architecture diagram is in the README.

Privacy

Video never leaves the phone. Names and progress stay on the TV. No account, no ads, no tracking. Privacy policy: https://yannegqpb2hvrix2nmxxbbygqm0ldexn.lambda-url.eu-central-1.on.aws/privacy

What's next

Spotter is already live on the Amazon Appstore. Next: Train Together, where two households start a workout at the same moment from a shared code, see each other's progress on screen, send preset cheers from the phone, and hear the coach say how their partner is doing. Still no account, no chat and no video. Also: Bedrock for the AI coach once our account is enabled (one setting away), more exercises (push-ups, planks with a timer), and testing on a real Vega Fire TV.

Challenges we ran into

  • No logs from the app on the Virtual Device. JavaScript console.log never shows up in vega device start-log-stream, so we debugged from screenshots.
  • No on-screen keyboard on the Vega Virtual Device, so typing a name with the remote was untestable. We moved name entry to the phone ("Send to TV").
  • WebSocket messages sometimes arrive split in two, which crashed JSON parsing and restarted the app; the TV now joins the pieces until they parse.
  • Keeping the AI honest. The coach's rewrites are checked against the workout's own numbers; one bug (a number written as a word, "Two reps") was caught while making the demo video.

What we learned

Building remote-first is its own discipline: every screen needs a clear focus target readable from 3 m. And the best coaching moment wasn't the AI summary but the rest between sets.

Full friction log and product feedback are in the repo (docs/friction-log.md, docs/product-feedback.md).

Demo video

Every app screen in the video is a real capture of Spotter running on the Vega Virtual Device. The opening living-room shot is AI-generated (Google Flow), with the real app capture placed on its TV. The narration is my own voice, cloned with Cartesia.

Built With

Share this project:

Updates

Submission history