FireLens AI — Generative Co-Pilot for Fire TV

An edge-resilient TV co-pilot adapting narrative depth to households (kids vs cinephiles), computing real-time sports win probabilities, and processing voice queries via AWS Bedrock.

Comment

Inspiration: Moving Beyond the "Prime Video X-Ray" Trap

Over a decade ago, Amazon revolutionized television with Prime Video X-Ray. It was a massive leap forward for answering two specific questions: "Who is that actor?" and "What song is that?" by looking up static records in IMDb’s relational database.

However, television is fundamentally a shared, 10-foot living room experience, and streaming still treats viewers with one-size-fits-all passive playback:

  • X-Ray cannot adapt to who is watching: An intense drama shows the exact same graphic trivia whether a 6-year-old child or a film student is on the couch.
  • X-Ray cannot reason about narrative causality: It cannot answer "Wait, why are they fighting and what did Tom just discover?" It only knows the actor's filmography.
  • X-Ray has zero predictive sports analytics: It cannot compute live win probabilities or explain why a referee blew a whistle on a loose-ball foul.
  • X-Ray is read-only: You cannot ask it unscripted questions or have a conversation from the couch.

When Amazon and Callstack launched Vega OS alongside Fire OS and challenged developers to build creative, AI-enhanced viewing that adapts to household patterns, we asked: What is the true generative successor to X-Ray?

The answer is FireLens AI—an edge-resilient, living room AI co-pilot powered by AWS Bedrock that understands narrative context, adapts to household profiles, and analyzes live broadcasts in real time.

What It Does

FireLens AI transforms passive watching on Fire TV and Vega OS into an interactive, household-adaptive viewing companion:

1. Household-Adaptive "Catch Me Up" (Narrative Persona Shifting)

Powered by AWS Bedrock (amazon.nova-pro-v1:0 and Anthropic Claude 3.5 Sonnet), FireLens AI dynamically adapts narrative depth to who is sitting on the couch:

  • 🧒 Family / Kids Mode: Automatically isolates and filters mature themes and spoilers, rewriting complex plots into engaging, age-appropriate story recaps.
  • 🎬 Cinephile Mode: Deep-dives into directorial choices, lighting schemes, scoring motifs, and thematic callbacks.
  • 🍿 Casual View: Fast 3-bullet catch-up ("Why are they fighting?", "What did Tom just discover?").

2. Live Sports Tactician Mode (Dynamic Win Probability & Live Telemetry)

Engineered with a real-time mathematical sports analytics pipeline:

  • Live Broadcast Feed: Ingests real-time arena box scores, shot clock, possession shifts, and referee reviews via REST/WebSocket (POST /api/sports/telemetry/ingest), dynamically recalculating continuous logistic win probabilities: $$P = \frac{1}{1 + e^{-1.45 \cdot Z}}$$ where $Z$ represents the score differential standardized by the game's time-decay variance: $$Z = \frac{\Delta_{\text{score}} + 1.2}{11.5 \cdot \sqrt{t_{\text{remaining}} / t_{\text{total}}}}$$
  • AI Referee Verdicts: Explains controversial calls (e.g. loose-ball foul vs. legal screening) right on the TV screen before commentators even reach the replay.

3. Interactive "Ask Bedrock" (Voice & Spatial TV Keyboard)

Viewers can ask unscripted questions about the scene via Amazon Fire TV App-Directed Voice Controls (ADVC), Web Speech API in the TV simulator, or a 10-foot on-screen D-pad keyboard. Queries route to an isolated AWS microservice with zero client-side credential vulnerability.

4. Edge-Resilient Fallback (Zero-Downtime Living Room Experience)

In a living room, buffering spinners are unacceptable. If the cloud connection drops or the backend is killed, FireLens AI instantly shifts to a local Edge Hybrid Engine (⚡ EDGE CO-PILOT • 2ms • 0 Tokens Billed) so playback never freezes.

How We Built It

Built With

Share this project:

Updates

Submission history