ContexCue

Inspiration

I wanted to solve a familiar streaming problem without creating a new one: when someone looks away or returns after an interruption, searching the web for a recap or a character can reveal the very next plot point. Rewinding is slow, and a generic AI answer cannot know exactly how far that particular viewer has watched. ContextCue makes the playback position the boundary of what the system is allowed to know.

What it does

ContextCue is a remote-first Fire TV experience for catching up on a film or identifying a character without spoilers. A viewer pauses, selects Catch me up or Who's that?, and receives an answer grounded only in scenes at or before the current playhead. The on-screen Watched / Upcoming bar makes that boundary visible.

After a recap, the viewer can choose More context, Keep it shorter, or Explain more simply with the D-pad. “More context” is not a paraphrase of the previous answer: it must explain the current moment and connect it to at least two earlier watched scenes. The viewer’s anonymous style choice is remembered for the session; story facts and previous answers are not stored in memory.

How I built it

The submitted app is React Native for Vega OS with the Amazon W3C Media Player, tested in the Vega Virtual Device. The player provides the exact playhead. The TV app sends only an asset ID, action, and playback boundary to API Gateway; it never sends a client-controlled timeline or future plot data.

For ingest, ContextCue parses timed WebVTT or approved catalog metadata into a versioned cue manifest. Each segment is summarized with Amazon Nova Lite and validated before it is published privately to Amazon S3. At playback, AWS Lambda derives the exact manifest prefix at the playhead. A ContextCue Temporal Agent, implemented with the Strands Agents SDK and Amazon Bedrock, must call a read-only temporal tool before responding. An independent Spoiler Guard then checks answer length, required watched-scene grounding, visible-character validity, and future-entity leakage before text reaches the TV.

Amazon Bedrock AgentCore Memory stores only the bounded response-style setting after an explicit choice. If the API, model, or network is unavailable, the app still returns a reviewed on-device recap that respects the same boundary.

Challenges and what I learned

The hard part was not generating a concise recap; it was proving that it could not use information from later in the programme. Prompt instructions alone are not a sufficient safety mechanism. I learned to enforce the constraint in layers: filter context before inference, expose only request-scoped tools, validate structured output after inference, and keep a deterministic fallback.

I also found that a “More context” control needs a measurable product contract. An upper word limit alone lets a model return another short sentence. ContextCue therefore uses a 22–30 word detailed-recap range, checks that the answer covers the current scene plus two earlier watched cues, and preserves the viewer’s chosen style during a transient provider failure. The result is an AI-enhanced viewing feature that feels native to a TV remote while keeping the viewer in control of spoilers.

Built With

• accessibility
• agentcore
• agents
• ai
• api
• bedrock
• cloudwatch
• fire
• gateway
• generative
• lambda
• memory
• native
• natural
• processing
• react
• s3
• sam
• sdk
• strands
• streaming
• tv
• typescript
• vega
• video