SceneBridge

• 

  1. TV home screen

• 

  2. Answer withheld before the reveal

• 

  4. Evidence replay

• 

  3. Answer with supporting evidence

Inspiration

Missing one important moment can make the rest of a film confusing. Searching for an explanation can reveal the ending, while repeatedly rewinding breaks the viewing experience.

SceneBridge explores a simple idea: help viewers understand the story using only information available before their current playback position.

What it does

SceneBridge is a TV viewing companion that lets viewers pause and choose “Recap last minute,” “Explain this scene,” or type a question.

Answers include timestamped evidence from earlier moments. Viewers can replay that evidence, then return to their original paused position.

Our original three-minute mystery, “The Last Screening,” demonstrates the experience. Ask “Why did Mira hide the key?” before the explanation is revealed, and SceneBridge withholds the answer. Ask afterward, and it provides the relevant passage with a replayable reference.

How we built it

The native TV application uses Kotlin, Compose for TV and Media3. It supports D-pad navigation, text entry, seeking and saved playback progress.

A TypeScript and Fastify backend captures the question’s playback timestamp and excludes transcript passages that end after it. Evidence references are checked against those eligible passages before reaching the player.

We created the original illustrated mystery using Python, Pillow and FFmpeg, with offline eSpeak-NG narration and timed captions.

The demonstrated version uses clearly labelled offline transcript extraction. Separate AWS integrations are implemented for Bedrock, S3, Polly and Transcribe, but live AWS operation has not yet been verified.

Challenges we ran into

The hardest part was preserving the original question timestamp while allowing viewers to replay earlier evidence. Rewinding also needed to discard later context.

TV development introduced challenges around remote focus, screen layout and emulator setup. Reviewing the native recording exposed a player threading issue that initial assertions missed. We fixed it by explicitly running player operations on Android’s main thread and requiring actual playback in the interaction checks.

Accomplishments we're proud of

We delivered an installable native TV prototype, an original narrated mystery and a working evidence-replay flow.

All 27 automated tests pass across backend behavior, browser interactions, Kotlin playback logic and native TV interactions. Android build and lint checks also pass.

What we learned

Playback position needs to shape the entire answer pipeline, rather than appearing only as a prompt instruction.

We also learned that passing assertions is only part of verification: watching the actual device recording revealed a problem that functional checks had overlooked.

What's next for SceneBridge

Next, we want to verify the live AWS pipeline, evaluate generated answers for grounding and spoiler leakage, and support additional licensed videos.

Future improvements include voice input, multilingual explanations and testing with viewers on physical Fire TV devices.

Built With

• amazon-fire-tv
• android-tv
• built-with-?-add-these-as-individual-tags:-kotlin
• espeak-ng
• fastify
• ffmpeg
• jetpack-compose
• kotlin
• media3
• node.js
• pillow
• python
• typescript