WordWatch

WordWatch a Fire TV (Vega OS) app for the Amazon Developer Hackathon 2026. Pause a film, move a cursor across the underlined subtitle words with the D-pad, get an in-context meaning

Comment

Inspiration

Looking up a word while watching something means grabbing a phone, missing the scene, and losing the thread. E-readers solved this years ago with Word Wise, but nothing like it exists for TV. WordWatch brings that lookup to Fire TV: pause the film, move to a subtitle word with the D-pad, and get its meaning, a translation and a pronunciation without leaving the couch or putting down the remote.

How we built it

  • App: React Native for Vega (Vega OS), in strict TypeScript. One reducer holds the app state, and the logic is pure functions with Jest tests (cue lookup, span navigation, quiz building). It uses only @amazon-devices/* packages, React and React Native.
  • Content pipeline (tools/pregen): parses the subtitles of the CC BY film Tears of Steel and uses Amazon Bedrock (Nova Lite) to annotate the cues and build a lexicon. Output is checked against JSON Schemas shared with the backend. Character offsets are computed locally and never taken from the model, so an underline always lands on the right word.
  • Backend (services/api): an AWS SAM stack with API Gateway (HTTP API), Lambda, DynamoDB as a TTL cache, Polly for audio, a private S3 bucket with presigned URLs, and least-privilege IAM.
  • Offline first: the base Word Card always renders from the bundled data, so the UI never waits on the network. A live call gets one attempt with a timeout of $t = 6\,\text{s}$, then falls back to "Needs a network connection". Offline, Spanish falls back to the bundled translations.
  • 10-foot UI: everything works with Select, the D-pad, Back and Play/Pause. Type is large, focus shows as a ring plus scale, and there are no scrolling lists.

Challenges

  • Remote input: Select arrives as enter or kpenter, with key-down and key-up as separate events. Acting on key-down meant a newly focused word received the key-up and opened by accident, so every Select acts on key-up.
  • Focus: default D-pad focus movement can't be overridden, so the layout is built around focus rows and TVFocusGuideView traps.
  • Media: media URLs must be https or a bare local path, so the film is bundled with the app on the Vega Virtual Device. Seeking there freezes the frame for about 5 s.
  • Tooling surprises: vega run-app silently wipes app storage, and a scripted remote needs about 1 s between key presses. Each surprise is logged with the time it cost in our friction log.
  • AWS: the account is on a single-Region free plan, so we moved from us-east-1 to ap-southeast-2 and the apac.amazon.nova-lite-v1:0 inference profile. Bedrock then hit "account is being verified" and then a daily token throttle, even on a one-line prompt. As a result, the live backend is built and tested with stubs but not deployed. The submitted app runs on a 19-entry fixture lexicon and bundled translations.

What we learned

  • Design for the remote first. Key-up versus key-down, focus traps and remote-only flows shape the whole UI on a TV.
  • Never trust model output for structure. Let the LLM write the text, and compute anything positional or structural yourself.
  • Offline first pays off. Because the app never depended on the network, a blocked backend cost us features, not a working demo.
  • Check account limits on day one. The plan, the Region and the model quota decide what you can ship, and none of them show up until you make the first real call.

Built With

Share this project:

Updates

Submission history