Ambient Frames

Inspiration

The most-watched "app" on living-room TVs is a ten-hour fireplace video. It buffers, it repeats, it carries ads. A TV should be able to make that scene itself.

What it does

Five procedurally generated moods — Deep Space, Rain on the Window, Fireplace, Dawn Aurora, Midnight Ocean — with a sleep timer and a three-button remote flow. Auto learns the household: it starts from a time-of-day schedule, then watches what this TV's remote actually does. A mood someone picks by hand and keeps on screen for two minutes counts as a vote for that part of the day; an Auto pick switched away within five minutes counts against it. From then on Auto plays the household's choice without a key press and says why on screen ("Learned · you chose Rain on the Window on 3 of your last 4 evenings"). Down twice forgets everything it learned. Nothing streams. Each mood is a ten-minute seamless render that restarts when it ends, and the automatic schedule rotates so two evenings in a row don't open with the same scene.

How we built it

Content pipeline: every mood is a JSON file. render_audio.py synthesises the layers with sox (noise beds, drones, a soft binaural pulse) and render_video.sh renders the visuals with ffmpeg, muxed into one MP4 per mood. Re-run the script, get a new render. Engine: a pure TypeScript reducer (day-part → mood, pin/auto, sleep timer, household learning) with 17 unit tests. Learning is counting, not a model: one vote per mood, day part and calendar day (ten presses in one evening are one vote), a 28-day window, and Auto only re-decides when the day part changes, so a vote never yanks the scene mid-evening. Storage: the history (at most 200 small records: day, day part, mood, pin/skip) lives in AsyncStorage on the TV and never leaves it; if storage fails the app keeps learning in memory for the session. App: React Native for Vega with the W3C media element (@amazon-devices/react-native-w3cmedia), D-pad focus navigation, running on the Vega Virtual Device.

Challenges we ran into

The Vega media element has no loop/volume, so live audio mixing on device is out; the mix moved into the render step and the app restarts the file on ended. Linux virtual devices have no WebView, which ruled out a web shell.

Accomplishments that we're proud of

Zero streamed or licensed media: five 1080p scenes and their soundscapes come out of two scripts (render_video.sh, render_audio.py), loudness-matched to -20 LUFS and checked for flashes by check_render.py before they ship. Household learning that is honest: counting, not a model. It explains itself on screen ("you chose Dawn Aurora on 3 of your last 3 evenings"), never switches a scene mid-evening, and two presses wipe it. Verified end to end on the Vega Virtual Device: install, playback, remote keys, learning across relaunches, all confirmed from device logs before a single frame was filmed.

What we learned

Read the platform before designing around it. The Vega media element lacks loop and volume, so we moved the mix into the render step; that constraint gave us seamless files and a simpler player. On a TV, "smart" must stay legible. Every Auto decision needed a one-line reason, or a scene change looked like a bug. Judge by what the device does, not by what the code says: a minute tick was rewinding video every 60 s and only the VVD log showed it.

What's next for Ambient Frames

User-authored moods (edit the JSON on a phone, render on the TV), Alexa+ voice pinning ("Alexa, fireplace"), a YouTube ambient channel fed by the same pipeline.

Built With

• ffmpeg
• react-native
• sox
• typescript
• vega-os