RelayLab

What RelayLab is

RelayLab is a focused B-roll overlay editor built around human-agent collaboration. A creator keeps control of the visual timeline while an external AI agent works through WebMCP to understand the transcript, inspect indexed B-roll moments, propose ghost edits, respond to human changes, and commit only after explicit approval.

The base talking-head video stays locked and remains the master audio source. B-roll is always visual-only and muted. The agent can propose, revise, remove, search, and replan, but it cannot approve its own plan, change human locks, or trigger paid video generation.

Why RelayLab is a strong fit for WebMCP

Traditional browser automation gives an agent pixels, DOM controls, approximate coordinates, and brittle click sequences. RelayLab instead exposes the actual editing concepts the agent needs: transcript ranges, source moments, timeline positions, pacing opportunities, ghost overlays, human locks, approval state, and the exact tools currently allowed.

This makes the agent state-aware rather than screen-aware. It can reason about the same structured timeline the human is editing and safely adapt when the human changes something.

Better user experience

Instead of asking a creator to manually hunt through long B-roll reels and place every clip from scratch, RelayLab lets the agent do the repetitive search and planning work while preserving human editorial judgment.

The collaboration loop is:

1. The agent reads the transcript, timeline, pacing opportunities, and indexed source moments.
2. It proposes ghost B-roll edits on the same timeline the human sees.
3. The human can move, swap, resize, remove, or lock any edit.
4. The agent rereads the updated timeline and replans only unlocked sections.
5. The human approves the plan.
6. Only then does commit_approved_plan become available through WebMCP.
7. The agent commits the approved plan without overwriting human timing, locks, or source choices.

For moments where uploaded B-roll is a weak match, the agent can suggest an AI-generation prompt. Actual generation remains a separate human-clicked action, so the agent cannot spend credits on its own.

What people and agents can do together that was difficult before

RelayLab turns a video editor into a shared stateful workspace between a human and an AI agent. The agent can inspect the real editorial state, understand source-time versus timeline-time, preserve locked human decisions, avoid re-proposing explicitly rejected moments, and replan around live human edits.

That is difficult to do safely with normal browser automation because the agent would need to infer editorial state from the UI and could easily overwrite user decisions. With WebMCP, the collaboration contract is explicit and bounded.

How WebMCP is implemented

RelayLab registers its tools through document.modelContext.registerTool(...) in a centralized WebMCP layer. The tool surface changes with project state.

Always-available read tools include project summary, transcript, timeline, edit plan, overlay opportunities, and B-roll search. During planning, additional tools allow proposing, updating, removing, and replanning unlocked ghost edits. After the human approves in the UI, those planning tools are removed and commit_approved_plan becomes available. After commit, only the read surface remains.

RelayLab also tracks a monotonically increasing timelineRevision. Mutating tools can reject stale agent writes with STALE_TIMELINE instead of silently overwriting a timeline the human has already changed. Locked overlays reject agent mutation with HUMAN_LOCKED.

The project includes a deterministic /demo route with a pre-indexed transcript and B-roll moments so judges can test the WebMCP collaboration loop without API keys.

Additional capabilities

RelayLab supports local B-roll source reels, timestamped transcript segments, manual and automatic captions, uploaded-footage-first search, optional human-triggered fal.ai B-roll generation, edit JSON export, SRT export, and reproducible ffmpeg export commands. The base video's audio remains the only audio source.

Built for the WebMCP Challenge

RelayLab was created during the challenge period as a focused experiment in how a human and an external AI agent can safely co-edit the same creative artifact without handing full control to either side.

Built With

• fal.ai
• ffmpeg
• next.js
• openai-api
• playwright
• react
• typescript
• vercel
• webmcp
• zustand