Previously TV

AI recaps for movies and shows that adapt to your memory

Comment

Inspiration

Your favorite show is back. But after so many shows, the details of past seasons start to blur. Who betrayed whom? Why did they leave? And how did it all end?

The recap flies by, but you are still trying to connect the characters and events. A single recap has to serve everyone, even though each viewer remembers different parts of the story. Rewatching entire episodes takes too long, and searching online for an explanation can reveal what happens next.

We wanted to help viewers recall the story quickly through short videos of the moments they need to remember. A familiar face, a key conversation, or an earlier decision can bring back the context without requiring another full episode.

What it does

Previously TV is a personal AI recap builder for Fire TV. It turns moments from a movie or series into an interactive recap that adapts to the viewer’s memory.

The viewer can pause their movie or episode and open a full-screen feed of short clips from the original footage. The initial recap considers the available story across its timeline and presents selected moments.

As the viewer browses, Previously TV collects Remember / Don’t remember signals through simple behavioral and remote interactions. Moving on after briefly watching a clip signals familiarity; staying longer or returning to a previous moment requests more context.

When a viewer does not remember an event, Previously places related earlier events immediately after the current clip. For example, a confusing confrontation can lead to an earlier scene involving the same characters or story thread. The remaining queue then continues in order, while remembered moments are excluded from future selections.

An interactive story map helps viewers explore characters and their relationships, with portraits and available story details. Selecting a character or connection requests relevant clips.

Viewers can also open Ask Previously, a Nova-powered assistant, to ask questions about the story so far. Answers appear as text and can be spoken aloud, with follow-up questions supported.

How we built it

Our pipeline is: prepare the source media -> analyze frames and available dialogue with Nova -> save a timestamped action logbook -> assemble a personalized queue -> cut and play the selected footage.

Understanding the story:

We use Amazon Nova to analyze timestamped video frames alongside timed transcripts when available. Visual analysis also supports silent scenes and films without a transcript. The preparation pipeline samples frames at roughly two-second intervals and processes them in short batches. Nova identifies action beats, classifies their story importance, records supporting visual or dialogue evidence, and proposes recap edits. An optional second Nova vision pass reviews the retained footage and its captions. Validated annotations are stored in SQLite, including scene descriptions, evidence, story-thread tags and source timestamps. This creates a reusable action logbook for each source.

Building the personalized recap:

We separate shared story analysis from each viewer’s playback position, memory feedback, queue and history. For prepared sources, Previously TV selects clips from the saved logbook locally. Each memory signal does not require another LLM call. The selector excludes remembered events and previously selected footage, applies character or relationship filters, and finds related earlier context when the viewer needs help. The queue of recaps follows FIFO order. Requested context takes the next position ahead of unrelated pending clips, after which the existing sequence resumes.

Creating playable clips:

We cut the selected ranges directly from the original video. Each recap edit lasts 7–45 seconds and can combine up to four ordered source ranges to retain useful setup, action and outcome. Clips are generated on demand and cached. The frontend downloads each clip before playing it and preloads the next two, helping reduce interruptions on TV.

Designing for Fire TV:

The interface uses HTML, CSS and JavaScript, served by a Python backend and displayed through a Kotlin Android WebView wrapper on Fire OS. We built large focusable controls and D-pad navigation. The story map reads supplied character and relationship records from SQLite, while timestamp checks control which facts are revealed.

Amazon Nova assistant and AWS integrations:

The assistant retrieves relevant dialogue, saved action descriptions and available relationship facts before the playback cutoff. Nova receives this bounded evidence and conversation history, and factual answers must reference supplied evidence.

The project supports two inference modes: the Nova developer API and Amazon Bedrock, with Strands Agents SDK or Bedrock AgentCore Runtime execution. Amazon Polly and Amazon Transcribe integrations provide speech output and audio transcription. We also implemented S3 asset snapshots and SageMaker Processing tools for batch logbook preparation.

References:

Build with Nova - multimodal story analysis and conversational reasoning.

Strands Agents SDK - agent execution with Bedrock.

Tears of Steel, Sprite Fright, and Spring - open movies used for the prototype.

Challenges we ran into

  • Understanding what a scene actually proves: A line of dialogue can be ambiguous, and a plausible summary can still misrepresent an event. We added evidence references, timestamp checks, structured-output validation and bounded repair attempts. These checks improve grounding, although sampled-frame analysis can still miss brief actions.

  • Making personalization responsive: Reanalyzing footage after every interaction introduced unnecessary latency and repeated requests. We moved source understanding into a reusable action logbook, so prepared recaps adapt through local selection.

  • Working with TV hardware: Playback buffering and speech support differed across devices. We added clip downloading and preloading, plus multiple assistant entry points. The Alexa remote microphone belongs to Alexa, so app voice input requires a supported microphone path (for example, app on the phone); the interface also supports typed questions.

What’s next for Previously TV

  • Personalized discovery: Extend short-video selection to help viewers choose what to watch next. With separate controls for “I remember this” and “I’m interested in this,” Previously could learn which characters, themes or story styles attract a viewer. It could then assemble spoiler-safe previews from approved promotional footage and explain why a title may interest them.
Share this project:

Updates

Submission history