Family Fit Quest

"Level Up Your Living Room — Play, Move, and Fit Together!"

Comment

## What it does

Family Fit Quest is a multi-modal family fitness game and 10-foot remote spatial UI experience built specifically for Amazon Fire TV. It transforms standard home workouts into immersive, high-energy gaming quests for up to 4 family members simultaneously (kids and parents).

🎮 4 Gamified Quest Modes: 🌴 Jungle Dash: Dodge low-hanging branches by squatting, leap over muddy logs, and sprint through jungle paths. 🚀 Cosmic Dance-Off: Strike galactic poses aligned with glowing tempo markers in space. 🌋 Lava Temple Escape: Side-step rapidly left and right to dodge rolling temple boulders. ⚡ Superhero Academy: Charge up energy barriers with high knees and landing power punches. 📺 10-Foot Spatial UI & Remote Focus: Custom D-Pad focus mapping engineered for Fire TV remote controls with glowing Fire TV Orange (#FF9900) and Neon Cyan (#00F0FF) visual state cues. Includes a live Fire TV Remote HUD Simulator. 🤖 AWS Bedrock AI Recaps: Analyzes real-time joint precision and endurance to generate kid-friendly storybook post-workout summaries, individual achievement badges, and adaptive coaching advice using Claude 3.5 Sonnet and Amazon Nova. 🎙️ Alexa Voice Integration: Press the Alexa microphone button on the remote to give voice commands like "Alexa, summarize our workout" or "Alexa, make it harder". 📲 Coffee-Table Smartphone Vision Relay: Uses our @family-fit/pose-stream package to relay pose keypoint telemetry from a coffee-table phone camera over WebSockets, eliminating the need for a native built-in TV camera.

How we built it

Frontend & Spatial Navigation: Built using React 19 and Vite styled with responsive Glassmorphism Vanilla CSS optimized for 60fps canvas rendering on TV screens. Computer Vision & Telemetry: Integrated 17-joint node skeletal movement tracking ($x, y$ confidence coordinates) for up to 4 players simultaneously. AWS Bedrock AI Integration: Connected to AWS Bedrock Runtime APIs (anthropic.claude-3-5-sonnet-20241022-v2:0 & amazon.nova-pro-v1:0) to process multi-player movement data into JSON workout recaps and narrative stories. Open Source Package (@family-fit/pose-stream): Created a standalone MIT-licensed JS package under src/packages/pose-stream/ that captures pose keypoints at 30fps and streams payload data over WebSockets to Fire TV applications. Audio & Haptics: Native Web Audio Synthesizer for 10-foot UI spatial sound effects and speech synthesis for reading Bedrock recaps aloud.

Challenges we ran into

Fire TV Spatial Navigation without Touch / Pointer: Adapting standard web interactions to D-Pad remote navigation required building a spatial focus matrix so every menu element, modal, and button has clear visual ring states and key listener focus hooks. Camera Hardware Limitations on TV: Most smart TVs lack a built-in camera. We solved this by architecting the @family-fit/pose-stream open-source protocol so any secondary mobile device acts as a wireless pose sensor relay for the Fire TV host. Multi-Player Motion Telemetry Balancing: Tracking 4 simultaneous skeletal nodes without lag required optimizing keypoint calculations and running canvas redraws inside requestAnimationFrame.

Accomplishments that we're proud of

Seamless 10-Foot Remote Experience: Created an intuitive UI navigation system that feels like a native Fire OS / Vega OS application. AWS Bedrock Narrative Engine: Transforming raw joint accuracy scores into personalized, encouraging stories for kids (e.g., "Maya dodged 12 jungle logs with superhero reflexes!"). Decoupled Open Source Package: Released @family-fit/pose-stream so any developer building Fire TV fitness or motion apps can easily implement smartphone-to-TV pose streaming.

What we learned

Deepened our understanding of Amazon Fire OS spatial UX guidelines and optimizing canvas graphics for TV screen resolutions. Learned how to prompt and parse structured JSON responses from AWS Bedrock Claude 3.5 Sonnet for low-latency workout coaching summaries. Mastered low-latency telemetry streaming techniques for real-time web gaming.

What's next for Family Fit Quest

Physical Fire OS Native App Package: Packaging the web app as a hybrid Fire OS app for distribution on the Amazon Appstore. Wearable Sensor Sync: Adding smartwatch heart rate sync to combine pose accuracy with cardio metrics in AWS Bedrock AI recaps. Expanded Quest Chapters: Releasing seasonal quest packs (e.g., Winter Wonderland, Undersea Atlantis) and custom family tournament leaderboards.

Built With

Share this project:

Updates

Submission history