ROUGH//CUT

An in-browser video editor you and your AI agent can use together

Comment

Inspiration

Most AI-powered video editors today suffer from a fundamental flaw- they take too much control away from the user. As someone who has been editing videos for over 5 years, I did not like this approach. While there are multiple tasks in video editing that can (and should) be automated and abstracted away, I, as the human behind the production, should have complete creative control and be able to tweak everything to my liking. With the introduction of WebMCP, I realized that I now had a way to solve this problem.

What it does

ROUGH//CUT is an in-browser video editor that a human and any WebMCP-compatible AI agent can use together. Both operate the same timeline through the same editing commands, so the agent is not working inside a separate abstraction or generating an edit that the user cannot inspect. WebMCP makes functionality discovery extremely easy for the agent and it knows exactly what all it can get done with those tools, without having to overload its context by inspecting the full document.

Users can upload a video, generate a word-level transcript, edit them manually or via their agent in natural language, transcript, timeline, and inspector, and export the finished project as MP4, EDL, or SRT. ROUGH//CUT exposes WebMCP tools for cuts, trims, gaps, clip reordering, captions, text overlays, transitions, colour adjustments, transforms, speed changes, background music, silence detection, and much more. There are a total of 40 WebMCP tools in ROUGH//CUT.

Every agent action appears in the editor, updates the same project the human is viewing, and can be undone. If the human changes the project while an agent is working, version checks prevent the agent from accidentally overwriting those changes.

How I built it

I built ROUGH//CUT with Next.js, React, TypeScript, and Tailwind CSS, deployed on Cloudflare Workers. Project data and revision history are stored in Cloudflare D1, while uploaded videos, music, and generated media are stored in R2.

The WebMCP integration uses the native document.modelContext.registerTool API. ROUGH//CUT's WebMCP tools for are used for inspecting, processing, editing, and exporting a project. These tools call the exact same versioning system used by the human-facing controls instead of maintaining a separate agent-specific editing system.

Transcription is handled by Whisper Large V3 Turbo through Cloudflare Workers AI, with optional support for OpenAI's whisper-1. Word timestamps are stored with stable IDs so captions and transcript-based edits remain connected to the source material.

FFmpeg runs inside a Cloudflare Container and handles waveform generation, silence detection, audio processing, and final MP4 rendering. The browser preview and FFmpeg renderer share the same timeline calculations and a canonical 1920×1080 coordinate system so the final export matches what was shown in the editor.

Challenges I ran into

The hardest part was keeping the browser preview, timeline, and FFmpeg output consistent. Browsers and FFmpeg often interpret transforms, brightness, fades, audio timing, and crossfades differently. A change that looked correct in the preview could produce a noticeably different export, so I had to test each operation across both rendering paths. This was quite painstaking and ended up surprising me (negatively) more than once.

Building a proper timeline was another large challenge. Trimming a clip affects captions, transitions, waveforms, playback, snapping, and the positions of every clip after it. Adding explicit gaps, non-ripple trims, speed changes, linked audio, and drag-to-reorder behaviour made this much more than a collection of UI controls.

WebMCP also introduced some interesting constraints. Agents can request an upload or prepare an export, but browser security correctly requires the human to select local files and confirm downloads. I designed these interactions as visible human handoffs instead of attempting to work around those protections.

Finally, FFmpeg performance inside a cloud container was very different from performance in the local docker container I was running. I had to improve the filter graph, remove unnecessary processing, handle cold starts, and keep exports asynchronous without hiding their real status from the user or agent.

Accomplishments that I'm proud of

I am most proud that ROUGH//CUT is a fully working editor which promotes human <-> agent collaboration instead of handing off creativity to an AI. This is how I envision all other apps and tooling will end up behaving once WebMCP becomes mainstream and popular.

The human editor is fully usable without an AI agent, while the WebMCP tools are still capable enough for an agent to make precise changes across the entire project. Humans and agents do not have separate copies of the edit and share one source of truth.

I am also proud of the interaction between agent autonomy and human control. Agent actions are visible, reversible, and rejected when based on an outdated project version. The agent can handle repetitive work, but the human always retains the ability to review, adjust, undo, or continue editing manually.

Getting real 1080p MP4 exports, EDL files, captions, background music, transitions, and browser preview parity working across Cloudflare Workers and Containers was also a major milestone.

What I learned

I learned how revolutionary WebMCP as a standard can be. I had heard of it earlier while I was testing out on-device Gemini Nano but put off learning about it to another time. It feels like discovering a treasure that had been right in front of my eyes all this time.

I also learned that tool design matters as much as the model using the tools. Narrow schemas, structured responses, project version checks, and compact diffs made the agent more reliable than giving it one large, overly flexible editing function. I dogfooded ROUGH//CUT by asking an agent what felt off when it was trying to use it with me, and then made fixes accordingly.

The shared command layer became one of the most important architectural decisions in the project. When human controls and agent tools use the same operations, consistency, undo, persistence, validation, and testing become much easier to reason about.

Finally, I gained a much deeper appreciation for the gap between browser media playback and production video rendering. Frame timing, audio syncing, colour processing, aspect ratios, and transitions all require explicit handling if the preview is expected to match the exported file.

What's next for ROUGH//CUT

My immediate focus is improving rendering speed, especially for cold container starts and longer videos. I also want to make frame inspection more efficient for agents so as to avoid bloating their context. I will also try and continue improving the editor on smaller screens.

Beyond that, I would like to add more caption styles, more effects, transitions, audio controls, export presets, and more. I'm also trying to figure out ways to make it work completely locally. I want to give the user a way get the entire transcription->editing->rendering->export pipeline running on their own machine. BYOK already exists for the transcription tool, but it's not local and instead uses an OpenAI API key.

The longer-term goal is for ROUGH//CUT to become a practical example of how creative software can work with external agents, all the while staying open source and in-browser. The agent should not replace the editor. It should enter the same workspace, use the same tools, and collaborate with the person who is ultimately responsible for the creative result.

Built With

Share this project:

Updates