Inspiration
A blind viewer watching a foreign-language film needs three accessibility features at once. Audio description tells them what is on screen. Spoken subtitles read the translation they can't see. The VoiceView screen reader is how they operate the television at all. On Fire OS none of the three knows the others exist, so all of them speak together, on top of the film's own dialogue. Blind users report turning VoiceView off to escape it, which trades the ability to operate the television for the ability to hear it.
Underneath that collision is a supply problem. Roughly 7% of streaming content has audio description at all. Human description costs \$15–30 per finished minute and takes two to four weeks per hour of video. Ninety-three percent of blind and low-vision viewers call audio description essential, and forty-five percent say they can't find described programming.
So two problems are stacked on each other: most films have no description, and where one exists, the platform's own accessibility features fight over the same speaker.
What it does
Baton is an audio director for Fire TV. One component decides who speaks: the film's dialogue, spoken subtitles, AI-written audio description, or Baton's own spoken UI narration. It guarantees exactly one voice at a time as an invariant, not a best effort. A cue may start only if its measured duration fits inside the silence it was scheduled for. If it doesn't fit, it is dropped and the drop is logged. It is never played late or over dialogue.
The demo stacks every voice at once. The bundled clip is an 88.6-second excerpt of Sintel, dubbed into German with Amazon Polly neural voices (Daniel and Vicki) over Blender's music-and-effects stem. On top of the dub, Baton runs six audio-description cues, twelve English spoken subtitles, a spoken notice that the description is synthetic, and spoken narration on every button focus.
It held on real hardware. On a Fire TV Stick HD run, read from the device log, the director had zero overlaps. All six descriptions were spoken, and nine of twelve subtitles. Two were correctly declined because they arrived with no silence to speak in; one was lost to a startup bug that has since been fixed. There were 0 audio underruns.
The film ducks by as much as the scene needs. While a voice speaks, the film is lowered by an amount computed from how loud the film is measured to be at that moment, so the narrator stays clear in loud scenes and quiet scenes alike. Measured at the Stick's own output, the duck arrives as commanded (0.997 dB per dB, residual 0.64 dB RMS), and the voice sits 12.1 LU above the ducked film, against a target of 12.5.
Everything is operable eyes-free. Below the Rewind / Play-Pause / Fast-forward bar is one more row: Description, Description length, Spoken subtitles, Captions, Suppress description and Screen curtain. Every focus change is spoken through the director. Description is off by default and remembered per title. Between two voices the director leaves a 300 ms pause, so they never run end to start. When VoiceView asks to speak, Fire OS tells Baton within 2–24 ms (3 of 3 runs on the Stick), and Baton cuts its own voice and waits.
It writes its own description. An offline pipeline transcribes the dialogue with Amazon Transcribe, maps speech and silence from the word timings, and drafts a description for each silence with a video model that is shown only the seconds that description will be spoken over. The pipeline then renders each draft with Polly, measures the real audio, and shortens anything that won't fit its gap. Every cue reaches the device with a measured duration, not an estimate. That matters because two voice engines on the same Stick disagreed on one sentence by 918 ms. The drafter was built for Amazon Bedrock (Nova), but AWS declined Bedrock access for this new account, so it runs on Google's Gemini API (free tier) behind the same interface. Live on the demo clip it wrote 12 descriptions, fitted to the speech-and-silence map from the real Transcribe output, and checked against frames from each one's moment, none describes something that does not happen. (In that run the speech lengths were simulated, not rendered by Polly, to keep it free.) The demo still plays a hand-reviewed track of six cues: the model's runs vary, and it still makes small slips (it called a red-haired girl "pink-haired").
Every voice is synthetic, and Baton says so. There are no human voice actors and no voice clones. Baton speaks a disclosure once per session and shows it on screen for the first ten seconds of playback.
Sintel © copyright Blender Foundation | durian.blender.org, used under CC BY 3.0. No Blender or Sintel logos are used. The English subtitles are Blender's own, retimed to the dub.
How we built it
Baton was built from scratch during the Submission Period; the first commit is from 3 September 2026.
Stack: Kotlin, Jetpack Compose for TV (tv-material 1.0.0), AndroidX Media3/ExoPlayer 1.4.1, Android TextToSpeech and AudioTrack, the AWS SDK for Java v2. The target is a Fire TV Stick HD (Fire OS 7, Android 9, API 28).
Seven modules, four of them plain JVM with no Android on the classpath:
:director: the arbitration state machine,advance(nowMs, inputs) → Directive. It has 124 tests, including a property test that drives 1,300 randomised timelines (cues, dialogue, seeks, pauses, stalls) and asserts at every tick that no two voices are active.:ports: interfaces only, so the director runs against a fake speaker in tests and a realAudioTrackon the device.:player: the ExoPlayer adapter, with a customAudioProcessorthat ducks the film and meters its loudness inside the render chain.:speech: engine selection, rendering, a speech-clarity filter chain, loudness normalisation, and playback whose end is detected by observation, not a timer.:loudness: ITU-R BS.1770-4 written from scratch (K-weighting, gated integrated loudness, short-term, 4× true peak), because nothing on API 28 measures programme loudness.:pipeline: the offline description builder, with live adapters for Transcribe, Polly and Gemini (each run against the real service) and for Nova (built and tested; AWS declined Bedrock access for this account, so it has never made a live call). A spend guard stops any AWS run before \$2, \$10 cumulative, or 150 Nova calls; the Gemini drafter is capped at 120 calls a run. The build fails if a generated cue speaks over dialogue, refers to itself, or names a character before the film has.:app: the one-screen, eyes-free player shell.
Verification: 459 test executions (372 distinct tests), 0 failures, 0 Kotlin compiler warnings, 0 lint errors, on the last full run. The build and tests have also been reproduced from a clean clone. None of it needs a device, a network or an AWS account. Behaviour on the Stick is proven from its own log and screenshots.
Challenges we ran into
FRICTION_LOG.md in the repository has all twenty-one entries, each with task, steps, expected and actual result, severity, workaround and suggestion. The ones that shaped the build:
- No platform arbitration API (entry 1). Nothing on Fire OS lets one voice learn that another is speaking, so Baton had to become the arbitrator itself.
- The good voice fails silently (entry 9). The default engine (IVONA) returns
SUCCESSand produces no audio until VoiceView has been enabled once on that device, and reports it only through a deprecated callback. Baton picks an engine by counting its voices and falls back to SVOX Pico. - No way to measure how loud the content is (entry 10). Every API 28 candidate fails for a different reason, so we wrote BS.1770-4 and validated it against the EBU Tech 3341 reference signals and
ffmpeg. - The platform can't duck fast enough (entry 12). Broadcast guidance asks for a 15 ms attack. AOSP's mixer changes volume once per buffer, and the shortest buffer on this Stick is 16 ms. Amazon's Vega OS audio library already exposes the ducking controls Fire OS 7 lacks.
- No way to record the Stick's own sound (entry 15).
screenrecordis video-only on Android 9, andscrcpyaudio needs Android 11. The demo is filmed with a phone, which is the honest capture for a product about sound. - A starter-template dependency forced a toolchain upgrade for nothing (entry 8).
tv-foundation1.0.0 is now an empty artifact that still demands compileSdk 35 and Kotlin 2.1. Deleting it fixed the build.
Accomplishments that we're proud of
- An invariant that is proven, not claimed. 1,300 randomised timelines assert that two voices are never active together, and the suite separately checks that the generators keep reaching the hard cases, so it can't pass by testing nothing.
- The full stack ran clean on real hardware: German dub, six descriptions, spoken subtitles and the disclosure, with zero overlaps and 0 underruns.
- We built the instrument the platform withholds, and it caught our own bug. A debug-only output meter (the release build ships without it or
RECORD_AUDIO) showed that the Stick copies mono audio to both speakers at full level. Every duck had been 3 dB too deep, putting the voice 15.1 LU above the film instead of 12.5. After the fix it measures 12.1. - A from-scratch loudness meter that matches the broadcast reference to a tenth of a dB or better.
- A cold start that explains itself. The first launch renders all 50 voice files (20.8 s to picture) and says so on screen and out loud. After that, cached files bring the film up in 2.96 s.
- Clarity by measurement, not taste. The speech filter chain was scored with ESTOI against the real film bed and holds an edge over unprocessed speech at +6, 0 and −6 dB SNR.
What we learned
- The policy is the product. The same booking rule that keeps a description off a spoken subtitle also kept it off the German dub. Treating every "don't speak here" interval the same way turned out to be the right generalisation, not a special case.
- "The platform doesn't support X" is only convincing with the line that stops you. Every friction-log entry about AOSP or Media3 cites the file and line.
- The fix Fire OS needs already exists one generation over. Vega OS ships ducking with a mode, a depth and a ramp time. Fire OS 7 has no equivalent.
What's next
- Drafting on Amazon Bedrock. The Nova adapter is built and tested, and takes the same prompt as the Gemini drafter that runs today. AWS declined Bedrock access for this account until it has more usage history, so it waits for a later billing cycle.
- Descriptions good enough to ship unreviewed. On the demo clip the live drafter leaves some silences empty and makes small slips; the next step is scoring drafts against the frames before they reach a viewer.
- VoiceView's own announcements. Baton already yields when VoiceView speaks. VoiceView still speaks on its own at playback start and at each screen-curtain toggle, and no app can stop that.
- Controls that speak from the first second. The access row's phrases render behind the film, so a toggle focused in its first ~7 s is silent.
- Description frequency and emphasis, the two parts of the preferences work that were cut.
- Ask-about-the-scene: a spoken question about the current frame. The director already arbitrates the answer; nothing raises one yet.
- A description-quality evaluation harness.
Log in or sign up for Devpost to join the conversation.