Rerun

Learn a language from the films you already watch, with your Fire TV remote: replay any line, step into its words, and meet them again as film-moment flashcards.

Comment

Inspiration

I learned much of my English from films and series — on the TV, on my laptop, on my phone. While my English was still limited, I got stuck again and again: rewinding ten seconds overshot the line, and by the time I had looked a word up on my phone, the scene was gone. At home, one TV serves people at B1, B2 and C2 — and someone who speaks no English at all — so whenever one of us stops to learn, everyone else waits.

I asked five other learners how they watch. All five watch on a TV. Four skip at least three in four of the lines they don't understand, because pausing, rewinding and typing into a dictionary breaks the film, and none of them keeps reviewing the words they do look up. Three share the TV with someone at a different level:

"Every time I pause to look up a bit of slang, my wife sighs and says, 'why is watching a film so exhausting?'" — Tuấn D., 31, project manager

"Lots of times I want to stop and listen closely to a good line, but finding the remote to rewind ten seconds is a hassle, so I just let it go." — Minh Q., 23, IT student

(Quotes translated from Vietnamese.)

In Vietnam, where Rerun was built, listening (470) and speaking (461) are the weakest English skills (EF EPI 2025). The tools that turn a film into a lesson — interactive subtitles, click-a-word, replay-a-line — live in desktop browser extensions. Films live on the TV. So I built Rerun: the lesson where the film is, the remote as the language tool, and one screen that works for a whole household at different levels.

What it does

Rerun is a learning layer for Fire TV, built on Vega OS. It turns a film with a subtitle track into a lesson you control from the sofa.

  • The remote is a language tool. ◀ hears the line again (twice quickly: the line before), and Rerun remembers it. ▶ jumps to the next line. ▲ pauses and turns the line's learnable expressions into stops; OK opens a card with what the phrase means in this scene, in your language, with an everyday example. Save it, or tell Rerun you already know it.
  • Who's learning tonight? Every person in the household has a language and a CEFR level. Words that are new for you are underlined in your colour, and the library shows how hard each film is for you.
  • Watch together. Pick several people: one film, everyone's new words in their own colour, a translation line per language, a save button per person. Auto-pause follows whoever finds the film hardest.
  • Phone companion, no install. Scan the QR code on the TV. Saved words land on the phone instantly. Say the line on screen back to the phone and the TV shows a word-by-word score. Ask "is this rude?" about any line.
  • Daily Rerun. Spaced repetition on the TV, where every flashcard is a moment in the film: the words you saved (the clip replays, you recall the meaning, you grade yourself with the D-pad) and the lines you had to hear twice, as listening cards — the clip plays with the text hidden until you ask for it.
  • Nine learner languages: French, Spanish, German, Italian, Dutch, Polish, Portuguese, Indonesian and Vietnamese. The demo household is Camille and Hugo (French, B1 and A2) and Ana (Spanish, B2).

Voice, D-pad and screen work together: listen and navigate with the remote, read on the TV, speak to the phone.

How I built it

  • Fire TV app — React Native for Vega (Vega SDK 0.24, RN 0.83): the W3C media VideoPlayer for the film and for replaying single lines in the Daily Rerun, useTVEventHandler and BackHandler for the remote, TVFocusGuideView for focus, the splash-screen hooks. Built and tested on the Vega Virtual Device.
  • Shared core — a pure TypeScript package used by the TV, the phone and the server: subtitle parsing, line navigation with the "press twice for the line before" rule, a key map per player mode, per-viewer word marking, SM-2 scheduling and edit-distance shadowing scores.
  • AI before you watch — a pipeline sends the whole script to Gemini (gemini-3.8-flash, @google/genai) in two structured passes: what is worth learning in each line, then translations and in-scene notes per language, as JSON validated with zod. The model names each expression; deterministic code finds it in the line's tokens and drops anything that is not there, so the TV never highlights a word that does not exist (45 expressions, 0 dropped). Prepared tracks are committed: playback never waits on a model and the app runs without an API key.
  • Measured, not assumed — the film ships with human subtitles in eight languages, so npm run eval scores Rerun's translations against them with chrF: Spanish 72.2, Italian 70.0, German 69.8, French 65.4, Dutch 64.2, Polish 61.6, Portuguese 52.4, Indonesian 51.7.
  • Server and phone — Hono, node:sqlite and WebSocket pairing rooms; a Vite + React phone page using the Web Speech API, served over HTTPS on the home network so the microphone works.
  • Tested on the device — 85 unit and API tests in CI, plus end-to-end tests that drive the real app on the Vega Virtual Device through the automation toolkit, with the phone companion paired: the phone remote, speaking scores, lines heard again, saved words, live questions and the household are checked on both screens.

Challenges I ran into

  • Video that failed silently. Vega's URL-mode player rejects cleartext HTTP sources, and paths without a file extension, with MEDIA_ERR_SRC_NOT_SUPPORTED and no message — before making a request. Two hours went into ruling out codecs.
  • Apps killed for "not responding" after the virtual device sat idle: the real cause was its display sleeping.
  • The OK key arrives as "enter", not the documented "select"; and useTVEventHandler re-subscribes whenever the handler changes.
  • Highlighting words that are really there. Language models paraphrase. Letting the model judge and code verify fixed it.
  • Designing for a D-pad. Every action had to be one key from wherever you are — which is why ◀ is "again".

All twelve issues are written up in the friction log with steps, evidence, workarounds and suggestions.

Accomplishments that I'm proud of

  • The whole loop runs on Vega: watch → hear a line again → step into a word → save → it is on the phone in about 1.4 s → say the line → review the word and the line the next morning as clips.
  • Watch together: three people, three levels, two languages, one subtitle line that serves all of them.
  • Translation quality is measured against human subtitles, line by line.
  • The remote-first core is an open-source library with tests, generated from the exact code Rerun runs.

What I learned

  • On a TV, the most valuable learning feature is the simplest: hearing a line again, with one key.
  • Let the model judge, let code verify. Structured output plus deterministic anchoring turned a creative model into a reliable component.
  • Vega is standard React Native — my shared TypeScript packages bundled unchanged — with a few platform rules worth knowing early (HTTPS media, 960-unit layouts, enter vs select).

What's next for Rerun

  • Any film with a subtitle track: titles from the user's own library, and films in other languages.
  • Speaking practice on the TV itself when Vega offers speech recognition, or through Alexa.
  • Testing with learners and families on real Fire TV devices, then a store release.

Built With

Share this project:

Updates

Submission history