CUTROOM

INSPIRATION

Twenty years behind a camera taught me the cruelest math in filmmaking: shooting is fast, cutting is slow. A weekend shoot makes six hours of footage and eleven hours of work. So the footage waits. Every creator on earth has that folder - the one with the good stuff in it that never became anything.

I was the guy people paid to rescue those folders. The bottleneck was never taste and it was never the software. It was the first pass: watching everything, remembering what you saw, and deciding what earns a place. That part is pure attention, and attention does not scale.

CUTROOM exists so the folder gets a first cut instead of a funeral.

WHAT IT DOES

You hand CUTROOM raw clips and one line of intent - “a confident 30-second promo of an artist reviewing framed prints.”

Four AI film agents take the job, each with a real cutting-room role:

DIRECTOR turns your one line into an editorial brief: target runtime, pacing band, tone, opening strategy, closing strategy.
SELECTOR actually watches every frame of every clip through Gemini’s multimodal video understanding, and returns timecoded moments with shot size, motion, subject and a strength score. It benches dead shots - a beautiful empty frame is still empty.
EDITOR cuts. It builds an edit decision list against the Director’s intent, with a “why” column on every single cut, obeying working-editor law: cold open, wide-tight alternation, motivated cuts, never pad to length, end on motion.
REVIEWER watches the ASSEMBLED cut back - real video in, not a text summary - and gets exactly one bounded improvement pass, the way a real room gets one more go before the client sees it.

You watch the whole argument happen in a live cutting-room log. Out comes a watchable 24fps rough cut, plus the EDL as JSON and CSV so any editor can open any NLE and finish the job.

It is not a finished film. It is the first eighty percent, delivered unattended, with its reasoning attached. On the sample footage the whole crew finished a 21.6-second cut from 64 seconds of raw clips on a free Replit deployment, and the Reviewer signed it: ship.

HOW WE BUILT IT (criterion: Technological Implementation)

Built with Replit Agent in small, gated milestones - the commit history is the diary. The agent laid down the foundation (intake, the Director, the Selector, the Editor); the last milestones (ffmpeg assembly, the Reviewer, hardening, key-pool rotation) were built against the same acceptance gates and merged through the workspace’s GitHub integration. Deployed on Replit at a public replit.app URL.

Every brain in the running product is Google Gemini via the google-genai SDK - gemini-3.8-flash with a gemini-3.5-flash-lite fallback, both env-overridable. There is no other AI dependency in package.json, by design and by repo-grep receipt. The Selector and the Reviewer both use the Files API for real multimodal video understanding: the model watches the footage, and later watches its own cut.

Cutting and rendering are ffmpeg. Output is h264, yuv420p, 24fps, on a canvas that matches the tallest source up to 1080p and never upscales - because 24 is the law in my house, and a hackathon demo that ships at 30 tells a cinematographer you were not paying attention.

The architectural decision that mattered most: the crew never touches the timeline. Agents emit structured JSON only. All cutting is deterministic code. The Editor proposes an EDL; the server validates it - do the moment ids exist, do the in/out points fall inside the source spans, do the durations recompute - and the model gets exactly one re-ask if it fails. The Reviewer’s orders are typed (trim / extend / swap / drop / reorder) with a signed delta and applied in code, capped at five, invalid orders logged as skipped. LLMs decide; software executes. That boundary is why the output is a real file and not a hallucinated timeline.

THE CRAFT MOAT (criterion: Quality of the Idea)

The non-obvious part is not that an LLM can watch video. It is what you tell it to look for.

The agents’ system prompts are two decades of cutting-room habit written down as text: cold-open selection, wide-tight shot alternation, 2-6 second promo pacing against 4-10 documentary pacing, motivated-cut language, a NO DEAD SHOTS filter, never pad to hit a runtime, end on motion. Most AI video tools optimize for what is technically detectable. CUTROOM optimizes for what an editor would actually keep.

Anybody can call the same API. The prompts are the moat, and they were validated against real footage before a single line of the app was written.

WHAT WE LEARNED (the honest section)

Perception was never the risk. The contract was. On first contact Gemini described handheld product footage correctly and with plausible timecodes - “camera pans left to focus on the green astronaut,” “camera zooms out to reveal the physical frame on a desk” - and benched a dead shot unprompted. But with a schema referenced rather than included, it invented field names and returned “00:03”-style strings. Embedding the exact schema JSON in every request, with “numbers as plain seconds, never MM:SS,” made outputs schema-valid on every subsequent call. Even then, prose leaked back in: the first production Selector returned shot_size as “close-up” instead of the CU enum, so normalization had to live in code. Never trust a prompt to enforce a type. Validate server-side.

“Works in dev” lied at the platform level, not the code level. The published deployment returned “unreadable or corrupt” for every clip while dev passed cleanly. The deployment image did not inherit dev-only system packages - ffprobe was simply not there. Our own rule caught it in minutes: every acceptance gate runs against the PUBLIC url, never the preview. Any binary your app shells out to must be declared for deployments. This is the single most transferable lesson of the build.

Production resilience is about outage waves, not quota. We hit six consecutive 503s across six different API keys on the video path - model-side, minutes long. The first agent-built implementation shipped the happy path: one 5-second retry, then permanent failure. The real requirement is exponential backoff into the 90-second range with per-clip resume, and a Files-API upload is bound to the key that uploaded it, so upload and analysis must share one key per clip. Separately, one free-tier key is worth roughly one heavy day of video analysis - quota exhaustion presents as a wall of 429s that looks exactly like a broken app. The fix that finally held was boring: a pool of keys, one client per key, rotate on every 429 or 5xx instead of retrying the same key, bench a per-day-exhausted key for 24 hours, and re-upload the video under the new key because the file is bound to the key that uploaded it. One exhausted key must never be able to take the whole app down.

Agent-built code ships the letter of the prompt’s happy path. Replit Agent was genuinely strong - it generated corrupt-footage and over-runtime test cases on its own and caught a session-reattach bug before handing off. But robustness was never volunteered. It came from acceptance gates that ran against production and refused to advance. The gates caught in twenty minutes what a 503 wave would have exposed live in front of judges.

The agents inherited the craft laws better than expected, and exposed a design gap. Given a 30-second target and thin material, the Editor delivered a tight 17.6-second cut and explained why, rather than padding. The Reviewer’s watched-runtime matched the real render exactly. And it wanted to make a shot longer - but “trim” was the only word we had given it, so it described a lengthening as a trim. We added a signed delta_sec. The machines will honor your standards; you just have to give them the vocabulary.

POTENTIAL IMPACT (criterion: Potential Impact)

The people who need this most are the ones who cannot pay an editor: the solo creator, the small nonprofit with a phone and a story, the touring musician, the two-person agency. For them the choice today is not “good cut versus great cut.” It is “a cut versus nothing” - and nothing wins most weekends.

CUTROOM does not replace an editor. It replaces the empty timeline, which is the actual thing that stops people. And because it exports a standard EDL rather than a locked proprietary render, a real editor can pick the job up at hour one instead of hour zero. The tool hands off to a human on purpose.

WHAT’S NEXT

Music-aware pacing (cut to the track, not just to the clock). Per-scene color notes. And the feature I want most: a shot list for the reshoot - the crew telling you what footage is missing, before you strike the location.