Fire TV prime

Prime Video X-Ray reimagined for generative AI: pause any scene on Fire TV to decode dialogue, explore lore, and catch up with your remote

Comment

Inspiration

We’ve all experienced it: sitting on the couch watching a gripping film or series, turning to a friend or family member, and asking:

"Wait, who is that character again and what did they do in season 1?" "What did they just whisper? Did I miss an important plot clue?" "Can someone catch me up on the last 15 minutes without spoiling the ending?"

While Amazon Prime Video revolutionized living-room metadata with X-Ray, static cast lists and trivia cards can only answer predefined questions. We were inspired to elevate this concept into the generative AI era.

With Amazon Bedrock and Fire TV, we saw the opportunity to build CineLens AI: a dynamic, conversational living-room companion that understands exact scene timestamps, characters on screen, narrative arcs, and audio subtext—controllable entirely with a standard Fire TV remote control.

What it does

CineLens AI turns passive viewing into an interactive, intelligent multi-modal experience:

Instant Scene Explanation: Pausing the movie or pressing the remote's AI Lens button activates an ambient slide-over panel explaining current stakes, hidden visual clues, and dramatic tension without spoiling what happens next. Dynamic Character & Actor Deep-Dive: Identifies actors currently on screen, their character backstory, and notable filmography. 10-Minute Spoiler-Free Recaps: Summarizes immediate preceding plot points if a viewer stepped away or arrived late, with strict guardrails preventing future spoilers. Dialogue & Subtext Decryption: Translates idioms, quiet whispers, and symbolic references that viewers often miss. Soundtrack & Acoustic Telemetry: Details the background score, instruments, and how the audio amplifies emotional beats. 10-Foot Spatial Navigation: Built specifically for television displays (720p/1080p/4K) with safe-zone compliance and zero mouse or touchscreen dependency.

How we built it

[Fire TV Remote / D-Pad] │ ▼ ┌────────────────────────────────────────┐ │ Fire TV Frontend (10-Foot UI) │ │ - Spatial Navigation Engine │ │ - Playback Telemetry Tracker │ └──────────────────┬─────────────────────┘ │ HTTPS POST ▼ ┌────────────────────────────────────────┐ │ Amazon API Gateway │ └──────────────────┬─────────────────────┘ │ ▼ ┌────────────────────────────────────────┐ │ AWS Lambda (Python 3.12 Serverless) │ │ - Contextual Prompt Engineering │ │ - Timestamp & Narrative Injection │ └──────────────────┬─────────────────────┘ │ Converse API ▼ ┌────────────────────────────────────────┐ │ Amazon Bedrock │ │ - Amazon Nova Lite / Claude Models │ └────────────────────────────────────────┘

Challenges we ran into

Designing for the 10-Foot Experience: Standard web apps assume touchscreens or cursor precision. Adapting complex AI responses to a living room 10 feet away required strict adherence to TV safe margins (5%), high-contrast typography ( ≥ 4.5 : 1 ≥4.5:1 contrast ratio), and foolproof focus rings (#FF9900 Amazon orange glow). Preventing Spoilers Programmatically: LLMs naturally know full movie synopses. To prevent ruining future plot twists, we engineered negative constraints and timestamp boundaries in the Bedrock system prompt, restricting responses strictly to: Timestamp  𝑡 ≤ 𝑡 current Timestamp t≤t current ​

Remote Control Latency & Focus Trapping: When opening the slide-over AI drawer, focus needed to transfer seamlessly to the first inquiry chip, and pressing Back (Escape / KeyCode 8 / 10009) had to instantly dismiss the drawer and return focus to the playback controls without navigating the browser back in history.

Accomplishments that we're proud of

Zero-Mouse Native Remote Navigation: The entire app can be driven blind using only a physical Fire TV directional remote or keyboard arrows. Dual AWS Deployment: Built with both an AWS SAM Serverless production stack (template.yaml) and a local hybrid fallback buffer ensuring 100% demo uptime for judges. 1-Click Review Experience: Anyone reviewing the code can run python run_demo.py and immediately test the full Fire TV experience with an interactive on-screen virtual remote HUD.

What we learned

Amazon Bedrock Converse API: The Converse API significantly simplifies multi-model switching, enabling seamless transitions between Amazon Nova and Claude models with zero prompt schema rework. Fire TV Web App Tooling: The Amazon Web App Tester proved to be an exceptionally rapid prototyping tool, dramatically reducing iteration time compared to traditional native APK compilation cycles. Ambient TV UX: Generative AI on TV must be non-intrusive. Glassmorphic slide-overs that retain visible video playback keep users engaged without breaking immersion.

What's next for Fire TV prime

Alexa Voice Services Integration: Enabling direct voice requests like "Alexa, ask CineLens who is speaking". Computer Vision Scene Recognition: Integrating Amazon Rekognition to automatically identify background props, filming locations, and wardrobe items with direct Amazon purchase links. Multi-User Living Room Voting: Enabling companion smartphone pairing via WebSockets so multiple family members can vote on what trivia or recaps to explore.

Built With

  • 10-foot-ui
  • amazon-api-gateway
  • amazon-bedrock
  • amazon-web-services
  • artificial-intelligence
  • aws-lambda
  • aws-sam
  • boto3
  • computer-vision
  • css3
  • fastapi
  • fire-os
  • fire-tv
  • generative-ai
  • html5
  • javascript
  • multi-modal
  • python
  • rest-api
  • serverless
  • smart-tv
  • spatial-navigation
  • uvicorn
  • vega-os
Share this project:

Updates

Submission history