Auteur

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Inspiration

Generative tools are getting blamed for slop, and I think the diagnosis is wrong. The output of a good model on a first attempt is usually fine. What makes work slop is that nobody went back for a second take.

They don't go back because iterating is slow or expensive. If nudging a gunshot forty milliseconds earlier means paying to regenerate it and waiting, you stop nudging. "Fine" becomes "shipped". The craft doesn't erode because the model is bad at its job. It erodes because being fussy got priced out.

So slop is not a quality problem. It is an economics problem. Auteur is an attempt to fix the economics rather than the model: make every act of taste free, and charge only for the one thing that actually consumes a GPU.

What it does

You give it a silent or under-designed video. A vision model watches it and proposes a Foley timeline: every sound it thinks belongs, pinned to the moment it happens. From there the entire interface exists so you can disagree.

Moving a sound is free. So is resizing, renaming, deleting, and re-ordering. Only generating costs money, and the app states the price before it spends and waits for consent. You can argue about placement for an hour without spending a cent, which matters, because placement is what decides whether Foley reads as real.

Director mode takes a note in plain language — "pull the gunshot back 80 ms", "move every footstep 40 ms earlier" — and answers with a proposal, not an edit. Old positions stay on screen, ghosted, behind the new ones. Nothing changes until a person approves it. The model has no commit rights. That constraint shaped most of the codebase.

Everything it produces is measured rather than graded. Integrated loudness and true peak against their targets, shot-to-shot mean-luma continuity with the numbers behind each comparison. It reports what it found. It does not tell you the film is good.

How I built it

Backblaze B2 is the source of truth. Not a bucket the app writes to on the side — the actual state store. Source video, scene analysis, every timeline version, stems, mixes, exports, QA reports, run records, the event log and the transaction ledger all live there. Completed generations are immutable.

The part I'd defend hardest: project status is folded from the event log rather than stored beside it. There is no status field anywhere that can drift out of agreement with what actually happened. Ask what state a project is in and the answer is computed from the record of events, every time. This also made the deployment boring in a good way — run state lives in B2, so polling is instance-agnostic.

The pipeline. Ingest to B2, then Gemini analyzes scenes and proposes a timeline of discrete, rhythmic and continuous events. A routing layer turns each event into a generation request. A planner walks the whole timeline up front and dedupes requests on (endpoint, prompt, duration), because several events routinely resolve to the same clip and asking twice would be two charges for one sound. Genblaze fans the misses out concurrently. Stems land in a content-addressed cache in B2, after which assembly is all cache hits. ffmpeg mixes against the timeline, muxes to the source, and measures the result.

One orchestration layer, three SFX tiers. Everything goes through Genblaze: a self-hosted MOSS-SoundEffect worker on a GPU box, ElevenLabs, and a fal fallback. Choosing between them is a config edit, not a different code path, and the paid tier falls back on provider error while the free testing tier deliberately refuses to escalate to a paid API.

Money is a first-class object. One immutable transaction row per call, recording the endpoint that actually served the work alongside the one that was requested, because fallbacks happen and the record should say so. Before any mix, a pre-flight probe prices only the cache misses, so a fully cached re-mix projects $0.00 honestly. When the probe can't reach storage the projection returns null rather than a cheerful $0.00 — an unknown cost is displayed as unknown. Per-project budgets are advisory and never refuse work; a separate daily USD cap on the paid tier does refuse, because those are two different instruments and conflating them would make one of them a lie.

Stack: Python 3.14, FastAPI, Svelte 5 control room served by the app, ffmpeg for all mixing and measurement, Gemini for analysis and director notes, ElevenLabs for sound effects, Genblaze for orchestration, Cloud Run for the API with a tunnelled GPU worker for the self-hosted tier.

Challenges I ran into

The GPU worker cannot live on Cloud Run. MOSS-SoundEffect needs CUDA and about 11 GB. The API runs on Cloud Run, the worker runs on a separate GPU process behind bearer auth, and it's reached over a tunnel. Splitting them meant the SFX tier had to become a genuine abstraction rather than a function call.

Deciding what a projection is allowed to say. The first version of the cost pre-flight returned zero when it couldn't probe the cache. That is the most dangerous possible bug in a tool whose entire pitch is cost honesty: it shows $0.00 for work that will bill. Making every projection field nullable, and teaching the UI to render "unknown", was a small change that took a long argument with myself.

Making the demo without lying. I wrote the narration script before filming, then discovered two lines claimed things the captured frames didn't show. One described the ghosted diff and approval flow; the capture harness is barred from sending a director turn because that bills a model call, so it only ever films the free surface. Another said cost is stated "first, and then it waits", but the shot was the Spend panel, which is the after picture. Both lines got rewritten to match the frames rather than the picture getting faked. Wrong audio is more expensive than an honest re-take.

The model's proposal was incomplete, and that turned out to be the point. Watching the finished mix, there were no gunshots. The proposed timeline ended at 11.26 s with two gun cocks and never fired, even though the duel plainly does. I measured five muzzle flashes off the source at 11.59, 11.80, 12.18, 12.61 and 12.88 seconds, added six events by hand across two prompts so the two revolvers wouldn't be the same sound repeated, and re-mixed. The pre-flight priced it at four cache hits and two misses: $1.08. Only the disagreement billed. That is the product's own argument running on itself, and I'd rather show it than a timeline that happened to come back perfect.

What I learned

The cache key is the cost model. Keying stems on (endpoint, prompt, duration) is what makes taste free, and it also means switching provider tiers regenerates everything under fresh keys. That's a real, permanent consequence of an early decision that looked like a detail.

Deriving state beats storing it. Every bug I didn't have this month was a status field I never wrote.

A green test run proves nothing about pixels. Twice I burned footage where frame count, duration and exit code were all correct and the video was black. Now the checklist ends with "open a frame from every clip and look at it."

What's next

Beyond Foley: the same shape — propose, approve, price, record — applies to dialogue cleanup, music beds and colour continuity. The modality is a parameter, not the architecture. Ahead of that, the honest list is per-event regeneration with seed pinning, a real diff view for timeline versions, and multi-user projects, which the event-log design should make less painful than it sounds.

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Auteur

Built With

• backblaze
• fastapi
• gcr
• genblaze
• python
• svelte