Everybody Moves

One family workout on Fire TV where each person gets their own version of every move: check in from your phone, and the coach adapts to everyone's level, energy and sore spots.

Comment

Inspiration

Family workouts fall apart in the first five minutes. One video can't fit everyone: Grandpa can't jump, Mom's knee is sore today, and the teenager is bored. So everyone ends up doing their own thing, or nothing. The TV is the one screen the whole family already gathers around. I wanted it to run one session where everybody moves together, but each person does the version that fits them.

What it does

  • Check in from your phone. The TV shows a QR code. Each person picks their name, rates their energy from 1 to 5, and marks anything sore (knees, back, shoulders, wrists). People without a phone join with the remote.
  • A plan for this household, today. The focus is whatever the family trained least recently. The length comes from the family's own history at this hour: when most recent evening sessions ended early, today is 8 minutes; when every one was finished, 15. The coach explains the plan out loud.
  • Same move, everyone's own version. During the workout, the TV shows a demonstrating figure and a countdown ring, plus one row per person with their own figure, version of the move, cue and reason: a glute bridge instead of squats for sore knees, a seated march, a chair squat and a couch plank for Grandpa, no jumping for anyone marked low-impact. A substitute must spare every sore area; if none does, that person sits the move out with easy breathing. The coach calls out only the differences ("Squats! Maya, glute bridge.").
  • Phones in sync. Each phone shows that person's current move, the same timer to the second, and captions of what the coach just said.
  • The remote runs the session. Play/pause pauses, fast-forward and rewind skip blocks (a skipped exercise doesn't count as done), back never drops you out mid-workout, and "End" needs a second press before it stops the session for everyone.
  • It learns. Everyone rates the session (easy, just right, hard) on their phone or with the remote. "Hard" lowers that person's level next time; "easy" twice in a row raises it. The summary says who changed and what the family's pattern means for the next session.

How I built it

  • Fire TV app on Vega OS with React Native for Vega (@amazon-devices/react-native-kepler 4.0, React Native 0.83), built and run with the Vega CLI on the Vega Virtual Device.
  • 10-foot UI: focusable buttons with hasTVPreferredFocus, useTVEventHandler for play/pause and skip, BackHandler for back. Layouts are designed at 1920 x 1080 and scaled to Vega's 960 x 540 layout units.
  • Coach voice: AudioPlayer from @amazon-devices/react-native-w3cmedia streams each line as MP3.
  • Exercise figures: drawn with @amazon-devices/react-native-svg from a few key poses per move, blended in code (no video or image assets, crisp at any size). The QR code is drawn the same way.
  • Service: a Cloudflare Worker with D1 for households, sessions, check-ins and ratings. The planner is deterministic, so the TV and every phone agree. Workers AI phrases the coach's introduction (Llama 4 Scout) and voices it (Deepgram Aura-2).
  • One clock: the TV, the service and the phones all compute the session position from the same start time, pauses and skips.
  • Tests: 29 for the planner (including the sore-area rules and skips) and the coach's guardrails, and 6 for the TV's coach cues, introduction and session clock.

Challenges I ran into

  • Making AI safe to put on a family TV. The plan comes from rules, not from the model; the model only words the introduction. In testing it still invented two family members ("Alex and Sam will get gentler versions"), guessed a pronoun from a name, promised a tired person "a gentler pace" (everyone shares one timer), and, in 6 of 8 runs, turned "sessions ended early, so today is 8 minutes" into "so we need a bit more time". Now the reason says which way the length moved, and the service throws a reply away if it names anyone outside the household, adds a number (in digits or words), says "longer" about a shortened session, talks about pace, leaves someone out, guesses pronouns, runs long, or takes over 8 seconds; then a template speaks instead. A re-run of the same households, plus one with a tired person who is also sore, came out correct.
  • Vega specifics. Layout units are half the pixels, which no one tells you until everything renders at double size. The QR-code library's install note (legacy-peer-deps) silently broke the build command. Several libraries listed for React Native 0.83 weren't in the 0.83 profile. Details are in FRICTION_LOG.md.
  • Recording a TV app with phones. I drove the remote with inputd-cli, recorded the simulator window and two phone-sized browsers, and placed the coach's audio from a log of exactly when the TV played each line.

Accomplishments that I'm proud of

  • Four people of different abilities in one session, each with their own version, in sync across the TV and two phones.
  • The coach is grounded: the plan and its reasons come from rules, the model only words them under checks, and the same words appear as captions on every phone.
  • It adapts on two levels: the person (ratings move levels) and the household (time of day, recent history).

What I learned

The hard part of a family product isn't the workout content. It's the moment where five people with different needs share one screen. Letting everyone check in at once on their own phone, instead of passing one remote around, and keeping the shared part on the TV made that moment work.

What's next

  • Test on a Fire TV stick, and remember the household on the device.
  • More moves and seated-only sessions for people who can't stand for long.
  • Alexa: "Alexa, start our workout" and voice check-ins for people without phones.
  • Weekly family goals.

Built With

  • cloudflare-d1
  • cloudflare-workers
  • deepgram
  • react-native
  • react-native-for-vega
  • react-native-svg
  • typescript
  • vega-os
  • w3c-media
  • workers-ai
Share this project:

Updates

Submission history