HomeBound: The Ambient Canvas Framework for FireTV

• 

  Picture 0: the original cartoon playing on Vega Virtual Device, before any note or companion interaction.

• 

  On Vega Virtual Device, Leo’s reminder appears during a commercial. Personal notes disappear when a guest joins.

• 

  Live Bedrock interpretation matches a recipe to the scene and updates it as the demo pantry is restocked.

• 

  A follow-up adds the jacket to a draft cart. Amazon Pay is unconnected; no order or payment is submitted.

• 

  Reading mode previews captions, muted audio and lighting while keeping the game. A follow-up restores the previous room.

Inspiration

A television is shared by a household and sometimes its guests. A useful personal note can interrupt a film or reveal private information to the wrong viewer. We wanted the living-room screen to help at the right moment.

What it does

HomeBound is a Fire TV canvas with an Alexa-style companion. It supports six everyday experiences:

• A homework reminder waits while an original cartoon plays, then appears beside a commercial. Pause to read and return to the saved playhead.
• A personal note disappears when a guest joins. The server removes ineligible notes before sending data to the screen.
• Ask what is cooking, see a matching recipe, and check ingredients against a demo pantry. Restocking updates the open recipe.
• Ask about a jacket in a movie scene, then say “add that to my cart.” It creates a draft with illustrative prices; no purchase is made.
• Keep the game on while previewing captions, muted audio, and a room setup for reading. Restore the previous preview with a follow-up request.
• Leave a note for Mom from the Alexa-style companion. The TV reveals the same note when Mom is the eligible viewer.

How we built it

The installable Amazon Vega application uses React Native for Vega and WebView. Its HTML, CSS, and JavaScript canvas plays our original H.264 cartoon and separate commercial clips. The demo is recorded from Amazon Vega Virtual Device on Apple Silicon, with the cartoon playing before Alexa or notes interaction.

A Python Starlette service owns the shared home state. SQLite retains notes, expiry, playback state, and event watermarks. Amazon Bedrock Converse interprets requests and the demo scene images using Amazon Nova Lite. Closed proposals feed bounded Context, Media, and Shopping components. Deterministic code owns audience eligibility, freshness, ordering, cart totals, and permitted state transitions.

The complementary Alexa-style interface uses the same backend and supports Deepgram speech in compatible desktop browsers. The video uses typed requests with the founder’s voice-over. Alexa+ partner access is not connected.

A separate Streamable HTTP MCP path uses Decionis AgentSafe for consequential device proposals, with signed evidence verified before simulated execution. The repository also supplies an ARM64 AgentCore supervisor container and CloudFormation for EventBridge V2, FIFO SQS, a Lambda bridge, published Bedrock Guardrails, encrypted logs, and alarms. These cloud deployment adapters are packaged and locally validated; no live AgentCore deployment is claimed.

Challenges and lessons

A shared TV needs server-side privacy, not just hidden cards. Viewing attention, audience eligibility, and purchasing authority are separate decisions around the model.

Vega packaging exposed practical issues: loopback HTTP needs an explicit allowlist; an automation runner must keep the simulator process alive; and a simpler, single-reference H.264 encode was needed for uninterrupted playback in our virtual device. We verified both commercial cues, pause/resume, and the return to the program. Our public friction log records the failures, working fixes, and product feedback.

What we accomplished

The same live canvas demonstrates audience-aware notes, pantry context, contextual shopping drafts, reversible reading previews, and shared state with a conversational companion. The release work includes a native Vega package, MIT-licensed source, pinned dependencies, Docker packaging, a deterministic source bundle, and a reproducible judging guide. The Python suite passes 204 tests, with 12 AgentSafe and 20 browser JavaScript checks also passing.

Current boundaries

Household presence, pantry, catalog, weather, and scene signals are labelled simulations. The cartoon is real in-app video playback; the other scenes are illustrated previews. Physical Ring controls, third-party streaming overlays, Amazon Pay, Alexa+ partner access, and physical room controls are unconnected. The configured managed Presence tenant connection remains unresolved. The live AWS integration demonstrated is Bedrock interpretation.

Next steps

Connect authenticated household signals, validate on physical Fire TV hardware, add household setup, and connect real device adapters through the existing execution authority. Extend media and catalog integrations with clear rights and purchasing boundaries.

Source, setup, and product feedback

• Source and MIT license
• Judging and setup guide
• Product feedback
• Verified status and limitations

Built With

• agent-safe
• ai
• ai-gent
• amazon-alexa
• amazon-bedrock
• amazon-vega
• decionis
• execution-authority
• fire-tv
• homebound
• iot
• mcp
• shopping