NPCrew

Inspiration

We wanted to bring the shared fun of board games and couch multiplayer to the TV: simple rules, quick turns, and people reacting to each other in the same room. Fire TV, phones, and a camera became our starting point for making the players part of the action.

What it does

NPCrew is a Fire TV couch-gaming prototype built around camera-based minigames and phone controls. Camera challenges map tracked body movements and poses to game actions. Our phone-controller and live-face experiments explore another way to participate: control the game from a phone and bring your face into the shared screen.

The submission focuses on these interaction foundations. AI-generated game mechanics and assets, richer cooperative games, and play between households are future directions.

How we built it

We used React Native and Expo for the Fire TV app, React and Vite for browser-based phone controllers, and TypeScript, Fastify, and WebSockets to connect the devices. MediaPipe provides body tracking, which we map into gameplay actions. We explored processing on the phone and on Fire TV, alongside live player-image streaming, to understand the responsiveness and device constraints of camera play.

The repository also contains experimental Amazon Bedrock/Nova and Polly integrations for themed content and host speech. These are separate from the camera-game demonstration; they do not generate the minigames shown.

Challenges we ran into

The main challenge was keeping camera input, streaming, and gameplay responsive across devices. We also worked through TV navigation, Windows native-build issues, and the difference between a game working on a development computer and being usable from the couch.

Accomplishments that we're proud of

Building a shared-screen prototype around body movement, phone controls, and player imagery. These interactions give us a practical foundation for games where the people in the room become part of what happens on the TV.

What we learned

Where computation happens affects responsiveness, but clear instructions and immediate feedback are just as important. A short, understandable interaction is a better foundation for couch play than a long feature list.

What's next for NPCrew

First, refine the phone-controlled face-cutout experience, test across more devices, and develop richer cooperative mechanics. Longer term, we want to explore AI-assisted creation of game content and assets, personalized challenges, and families playing together across households through their TVs. These are roadmap ideas beyond the current prototype.

Built With

• expo.io
• fastify
• fire
• firetv
• node.js
• react
• skia
• typescript
• vite
• websockets