Inspiration

Most living rooms have a big TV and nobody moving in front of it. Getting a family active together at home usually means buying something: a console, a camera, a subscription with its own hardware. Most TVs don't have a camera, but almost every household has a phone. We wanted the TV people already own to become a place where they play and train by moving, with nothing new to buy.

What it does

Fire Motion is a Fire TV app you control with your body. A phone propped up in front of the sofa is the camera. It tracks up to two people and streams their skeletons, with a small picture, to the TV.

The home screen has two carousels, driven entirely by the remote.

GAMES

Party Match (2 players): a random run of nine head-to-head minigames over one shared, mirrored camera picture, with cartoon art drawn on your bodies. You might catch eggs in a basket you hold, pop bubbles with a beak on your face, freeze when the owl looks, or duck to drop puppies onto a tower. Mini Games plays any one of them.
Blaster Bash (1 player): an illustrated shooter. Step left and right to aim and dodge; jog on the spot to reload.
Dash Duel (2 players): a 3D race. Hop over fences and crates, duck under saws, collect coins. If you're alone, the computer runs the empty lane.

WORKOUTS

Yoga Flow: guided yoga, stretching and breathing, with 28 poses in 9 sessions. An outline of each pose is drawn over you and turns green part by part as you match it. A hold only counts while you are still. Meditation sessions follow a breathing circle.
Fit Coach: bodyweight workouts, with 10 exercises, 5 workouts, and every exercise on its own. The camera counts every repetition, times every hold and coaches your form, one cue at a time.

Every rule is drawn on screen: the line a squat must cross, the outline of a pose, the line you duck under. Nobody has to guess what is being asked.

How we built it

The phone does the vision. The Android app (Kotlin, CameraX, MediaPipe Pose Landmarker) finds up to two people and decides who is player 1, the person on the left. It sends 17 joints per person in one small UDP datagram, and a JPEG of the same frame over TCP. A TV stick couldn't run a pose model at a playable rate; a phone can.
One Unity 6 project is the TV app, for both of Fire TV's operating systems:
- Vega OS: it runs as a web build inside a small React Native for Vega app's Vega WebView. A web page can't listen for the phone, so there it connects out over a WebSocket carrying the same bytes. The Vega app relays the remote's Back and Play/Pause keys to the game, and closes through BackHandler.exitApp() when the game asks.
- Fire OS: the same project builds an APK for both 32-bit and 64-bit devices.
Tested on the Vega Virtual Device: about 60 frames a second with the live camera picture, in a 182 MB WebAssembly heap. Scripts boot the device, serve the build, press remote buttons and take screenshots. The Amazon Devices Builder Tools MCP server answered Vega questions from inside our coding assistant as we built.
Measurements that hold up for real bodies. Everything is measured in the player's own body lengths, so it reads the same near or far, from the front or the side.
- Counting a rep. For a squat, the signal is how far the hips have dropped from where you stood, in torso lengths $L$: $$s_t = \frac{h_0 - h_t}{L}$$ A rep counts when $s_t$ rises past $\theta_{\text{down}}$ and then falls back below $\theta_{\text{up}}$, with $\theta_{\text{up}} < \theta_{\text{down}}$, so a wobble at the bottom never counts twice. When your feet are out of the picture, as with a laptop on a desk, squats and lunges are counted from your shoulders instead.
- Matching a pose. Yoga Flow compares the direction each body part points, $\hat{u}$, with the pose's, $\hat{v}$, either way round and on either side, so a mirrored or side-on pose still counts. A part matches when $$\arccos(\hat{u} \cdot \hat{v}) \le \tau$$ and the tolerance $\tau$ is wider in beginner sessions than in advanced ones, which ask for the exact pose.
- Every title was tuned against recordings of real people, and more than 500 automated tests pin the rules.
Private by design: poses and pictures only travel between the phone and the TV on your own network. No server, no account.

Challenges we ran into

Getting to Vega at all. The Vega Virtual Device runs WebView apps only on Apple Silicon Macs, so development moved from Windows to a Mac midway.
Remote keys. Back arrives in a Vega WebView as BrowserBack, which a game engine never sees as Escape, so Back did nothing anywhere until the page relayed it.
Real bodies. Hips leave the picture when people stand close, players never stand still, and the camera loses a wrist now and then. Every control had to keep working when part of the body is missing, and give both players the same sizes wherever each one stands.
A first launch that stayed black. An app launched the moment the virtual device says "ready" never loads, so our tools wait for it to settle and relaunch if the page isn't requested.

We logged each one, with steps and suggestions, in our friction log.