CUTROOM

A virtual cutting room: NVIDIA Nemotron agents watch your footage, commit to a cut, build a real Shotcut timeline over MCP, screen it, fix it, and hand you the pickup list.

Comment

Inspiration

I've been a cinematographer for 30 years. Shooting is fast; cutting is slow. Every shooter I know has a drive full of footage that never got cut because the first assembly is the hardest part: staring at an empty timeline at 2 AM, scrubbing clips to find the moments. I didn't want an AI that spits out a montage. I wanted the room I learned in: a director with an opinion, an assistant who logs every clip, an editor with craft rules, someone who builds the timeline, and someone honest enough to say "that cut sags at shot 3."

What it does

You give CUTROOM raw clips and one line of intent ("A 20 second Reel for the print. Quiet, patient, end on the astronaut."). Seven chairs go to work:

  • Research decides whether anything needs looking up (platform length limits, genre conventions) and searches the web with Tavily.
  • Director (Nemotron 3 Super, reasoning on) commits to a point of view: pacing, target length, shot-length band, opening and closing strategy, what to look for, and what to avoid.
  • Selector watches every clip as timecoded contact sheets plus a half-second motion and loudness profile, and logs every usable moment and every dead shot.
  • Editor (Nemotron 3 Super) sequences the cut under explicit craft laws. Every EDL row has to name its motivation.
  • Assistant Editor (Nemotron 3 Nano, tool calling) builds the timeline in a real NLE format by calling MCP tools itself: create_project, add_clip, trim_clip, move_clip, get_timeline. Then melt renders it.
  • Reviewer: a screener watches the rendered cut and Nemotron decides "ship" or one bounded pass of executable orders, which the Assistant Editor applies to the live timeline over MCP.
  • Coverage (Nemotron 3 Nano) writes the call sheet for tomorrow: what footage the cut is missing, with shot sizes, moves, and roll times.

You get back the rendered rough cut (first and final versions), a .mlt project that opens in Shotcut so you keep cutting where the agents stopped, the EDL with the reason for every cut, the reviewer's notes, the research sources, every MCP tool call, and the pickup list. A typical three-clip cut costs about $0.05 in model calls.

How I built it

  • Nebius Token Factory serves every model through one OpenAI-compatible client. Every reasoning and tool-use decision runs on NVIDIA Nemotron: 3 Super for the Director, Editor and Reviewer verdict, and 3 Nano for tool calling, research planning and coverage. Token Factory has no NVIDIA vision model today, so footage is described by GLM-5.3-Flash and judged by Nemotron.
  • MCP: the Assistant Editor is a Nemotron tool-calling loop over shotcut-mcp, my MCP server for Shotcut/MLT. Every get_timeline result carries a check the cutting room computes against the EDL, so the agent acts on facts. Code then verifies the final timeline frame by frame.
  • Seeing footage without a video model: contact sheets stamped with source timecode, plus ffmpeg scene-change and RMS loudness per half second. Back-to-back moments of one take are joined, because an editor may trim a moment but never extend it.
  • Streaming for every model call (long vision reasoning outlived idle connections).
  • A spend guard I'd trust with my own card. Every call is priced at its worst case before it's sent and refused if it could cross the stop. The ceiling can't be lifted by config, and on hosts that wipe their disk the ledger is mirrored to durable storage and the app fails closed.
  • Node 22, Express, vanilla JS front end, melt, ffmpeg, Python for the MCP server.

Challenges I ran into

  • shotcut-mcp couldn't survive its own tools. Every call reloaded the project and minted new clip IDs, so an ID returned by add_clip was dead on the next call, and filters, titles and crossfades disappeared between calls. I rebuilt its state model, rewrote crossfades the way Shotcut itself writes them, fixed titles that rendered "INVALID", and added real-render tests checked pixel by pixel.
  • An agent that undid correct work. In one run the Assistant Editor built the timeline perfectly, misjudged its own check, and deleted every clip. The fix was to stop asking the model to grade itself: the cutting room now computes the check and hands it the verdict.
  • Moments too short to cut with. The vision model logged 2.5-second slices of continuous camera moves, so the Editor couldn't hit a 6-10 second shot band. Joining continuous takes took the Editor from an 8-second cut to a 20-second edit that hit the target exactly.
  • A reviewer that gutted the cut. It correctly noticed the sample footage held three different prints and dropped three of five shots. Now a review pass may drop one shot at most and must keep 60% of the target runtime.

Accomplishments that I'm proud of

  • The agents don't describe an edit; they build one, in a project file a human editor can open and keep working on.
  • Every decision is inspectable: the intent, the moments, the EDL with motivations, the tool-call trace, the review orders, the research sources.
  • It costs cents per cut, and it can't spend money it wasn't given.

What I learned

Agents get reliable when the tools tell them the truth. The biggest quality jumps came from changing what the agent sees (ground-truth checks, joined takes, motion and loudness profiles), not from prompting harder.

What's next for CUTROOM

Editor-chosen dissolves and titles (the MCP tools already support both), audio-led cutting, multi-camera sync, and letting the Coverage list drive a second pass once the pickups are shot.

What changed during the submission period (Aug 26 to Oct 30, 2026)

CUTROOM v1 (built for an earlier hackathon) was a Gemini-only pipeline that cut with ffmpeg. Between August 26 and October 30, 2026 it was rebuilt:

  • every reasoning and tool-use step moved to NVIDIA Nemotron on Nebius Token Factory;
  • new Assistant Editor that builds and revises a real Shotcut timeline over MCP;
  • shotcut-mcp repaired (persistent state, working crossfades and titles, new tools, tests);
  • new contact-sheet and motion/loudness Selector and screening Reviewer;
  • new Tavily research step, spend guard, durable ledger, streaming client, web app and replay demo.

Links

Built With

Share this project:

Updates

Submission history