Release Airlock

The agent investigates. You retain production authority.

Comment

At a glance. A deploy console where an agent reaches 27 WebMCP tools and cannot execute any of them; every write is a proposal a person decides on the same page, beside the evidence. Stage-gated surface (13 tools in Triage, 27 in Recovery), a plan tool for order-dependent fixes, provenance grading for writes justified by untrusted content, a live storefront that breaks and recovers. Verified: 229 unit tests, 144 hit-tested Playwright gates, 91 machine-checked scenario variants, Chrome webmcp-evals 24/24 live, Lighthouse agentic 4/4, 488 real-model runs. Found and filed against the spec: a host that both calls tools and drives the DOM can complete the page's human step itself (webmachinelearning/webmcp#288, with a one-file repro and a page-side mitigation verified against the host). Live: https://release-airlock.vercel.app · Repo: https://github.com/Sidharth1999/webmcp-airlock

Inspiration

An on-call engineer, mid-incident, probably awakened at like 4am by their PagerDuty, wants an agent's speed without handing it authority over production -- speed literally means less revenue lost to the business and less users impacted by outages. But we can't be careless about what agents are allowed to do with our systems.

This is not a hypothetical based on some research. In the last fourteen months there have been nine documented cases of a coding agent destroying data, most of them with a permission system switched on that the agent talked its way past. Examples:

Every one of those permission systems lived either inside the model, which can be argued out of anything, or in a generic host confirmation, which knows the verb and the target and nothing about where the idea came from or what the operator is looking at.

Three more facts shaped the design.

  1. The cost of an incident is deciding, not clicking (Rootly).
  2. Mitigation happens in a console, and the kill switch beats the rollback (LaunchDarkly).
  3. Stage-gated access is an established pattern, just-in-time and break-glass (IBM). And Chrome's own WebMCP security guide asks for exactly this shape: "it's impossible to guarantee safety inside of a large language model" (secure tools).

WebMCP puts the tool server in the page. For this problem, that location is the whole point. The page is where the operator, the evidence, and the levers already are, so the page can be the authority on what the agent may do.

What it does

Release Airlock is a deploy and incident console where an agent can reach every lever, and not one of them moves without you.

The console, the storefront, and the agent ledger after a seven-step response, checkout restored

It exposes 27 WebMCP tools on document.modelContext: six reads, a notebook the agent writes its findings into, nineteen propose_* writes, and propose_plan. A write tool's entire effect is to put a card in front of the operator. The world moves when a person decides, and not before.

The tool surface is a function of console state. Move the response stage from Triage to Diagnosis to Recovery and tools are registered and unregistered underneath the live session (AbortController, so a real host fires toolchange). Triage grants 13 tools. Nothing that touches production exists in Triage; it is absent, not gated. A withdrawn tool leaves a tombstone, and explain_surface tells the agent why its capabilities changed. The gate is re-checked when the human decides, against the stage the operator has set by then.

The page grades the evidence it served. Every read is audited into the same event log the page renders from. When an agent proposes rolling back a deploy id that appeared nowhere in the console's state and reached the agent only inside a customer-supplied log line, the card quotes the line, names the read that served it, and promotes the write to a two-key approval.

For the operator, an incident is a stitching problem. The hard incidents are the ones where the order you do things in decides the outcome, and no single screen says "shed load first." The agent is good at exactly that stitching, and the console lets you audit the stitching before you let it act:

  • One ledger. Every beat is a row on one spine: connect, each tool call, each finding, the plan, each step, what that step did. A tool call opens onto the exact bytes the agent got back, with the log position they reflect and a link to the console surface they came from.
  • A citation is a place. "Contention cleared (#42)" is a button that lands you on log line 42.
  • The agent objects before your click. Reach for a control it has ruled out and its reasoning appears beside that control. It counsels; it never blocks.
  • A plan numbers the console rows it will touch and prices every step.
  • Every executed write says what it did. When nothing moved, it says why: "No effect: scale to 9 has no effect, the autoscaler ceiling for api is 6."
  • The customer is on screen. A storefront breaks when the incident starts, shows the status post you approved at step 4 word for word, and takes payments again when the fix lands. The receipt reads 7 of 7 approved by you, 0 writes went round you.
  • While an agent host is attached, approval is a held gesture, not a click.

The console stands on its own without an agent. Every lever is clickable by hand, states its cost, and is worth having on a bad day.

A write that changed nothing says why: halted at the autoscaler ceiling, free headroom first

How we built it

  • One event log is the truth. The world is a pure fold over it; the UI, the tool results, and the metrics all derive from (events, world). There is no second source of state to drift.
  • A Web Worker owns the log; the main thread renders and forwards. The tests and the unattended driver invoke tools through the same execute path real WebMCP uses.
  • 27 tools, documented from the registration path itself. The surface reference in the repo is generated by walking the real registration code through every stage, so it cannot disagree with the bundle. Reads return pages under 1.2 KB with a log position on every answer. Writes return only a proposal id.
  • The gate is checked on both sides. One table decides what the agent can see; a separate table decides what the engine will execute; a test asserts they agree, because the engine must never trust that a tool was unregistered.
  • Determinism is enforced. No wall clock and no randomness anywhere in the simulator; a linter fails the build. Same seed, byte-identical replay.
  • Scenarios are compiled and verified, not asserted. 91 variants across four families are machine-generated and accepted only if a scripted correct run beats doing nothing and every declared trap costs more than doing nothing, probed to the same horizon.
  • 229 unit and property tests, 142 hit-tested Playwright gates against the real page, CI on every push. No backend, no API keys; it runs in the browser.

Challenges we ran into

UX. Vibe-coding the UX for the project was a huge uphill battle and required hours of ruthlessness and discipline on teaching AI how to effectively present information on a page. Release Airlock deals with so much system state information and computation over with with an AI agent and that density needed to be presented really well in juxtaposition with the actual layout of an incident console. There were at least 20 iterations that were roughly functional before we landed on the most satisfactory one.

AI frustration. Prompting is extremely taxing and simply aligning your vision with your AI is an uphill battle. It makes strange assumptions, gaslights you about completed work, and will sometimes even reject your judgment when it's the right call. It's important here to always have a bearing on the state of the project so you can actually assume authority on direction without the project getting derailed by your AI.

A paginated tool whose contract lived only in prose the model skims. The first paid model run opened every read with cursor 0, and three tools answered with a silently empty page. The model reasoned about an incident having seen no deploys, no logs, and no traffic. The fix was one sentence in the description: "Omit it for the newest page; there is no page 0."

Answer keys that named a number instead of a decision. A run capped the route at 70 req/s and shipped, correct and in the correct order, and scored wrong because the key named the exact cap the console's preset offers. Keys now state the constraint.

The real host run. The night before submission, GPT-5.6 in ChatGPT's in-app browser ran the live URL. Two things went wrong. A rollout halted at the fleet's autoscaler ceiling, which is the scenario's trap, and the page reported "nothing in the world moved" with no reason, so the agent scaled, which also cannot work at the ceiling, and spiralled. And the in-app browser, which both calls the page's tools and automates its DOM, clicked Approve itself. Both were fixed the same night: every executed write now carries an outcome with a reason, status exposes capacity and headroom and offered versus admitted traffic, approval is a held gesture while a host is attached, and the receipt records the gesture. The second one is bigger than this project; it is point 7 of the spec feedback in the repo.

Update, the morning of submission. With the held gesture in place, the same host, asked only to "investigate and resolve this incident", moved the response stage itself to unlock the Recovery tools; the console's own log shows the stage change between two of its tool calls, nobody at the keyboard. To rule out our own code I built a one-file page with two tools and one Unlock button that logs every input it receives. Labelled "Unlock (operator only)", the host would not press it across four prompts. Labelled just "Unlock", it pressed it on the next prompt, 43 seconds after the refused call, and reported the fix proposed. Every event was isTrusted:false, so a page can refuse synthetic activation on human-only controls, which the console now does. Filed against the WebMCP spec with the screenshots, the logs, and the repro page: https://github.com/webmachinelearning/webmcp/issues/288

Accomplishments that we're proud of

The sophistication of the UX. Countless hours were spent nailing down layout, the agent panel UX and how to present information effectively over a tight space (an agent's investigation ledger is just so verbose). A lot of minor things were polished to really refine the UI as well.

What a person and an agent can do together here that they could not before: a human can let an agent reason all the way to a production change and still be the only thing that moves it, because the page is the authority on both.

  • propose_plan takes an ordered sequence and the reason the order is load-bearing. It is deliberately not a batch approval: step N+1 is not even proposed until step N has executed, so you always decide against the world as it is.
  • In the filmed incident, a retry storm that sustains itself after its trigger clears, the answer is two levers in one order. Cap the route, then ship. Reversed, the same two levers cost more than doing nothing. Over 24 verified variants: nothing 154.10, cap then ship 12.43, the full seven-step response 9.10, ship then cap 170.61, silence alerts then ship 573.75 with the api down in 24 of 24. A runbook cannot encode a cost that depends on what you do next.
  • It was run, not just tested. 488 scored real-model runs with and without the gate: zero catastrophic outcomes in either arm, zero of 392 gated writes executed without a decision, and in the ordering family only gated runs got the order right (4 of 24 against 0 of 24). Caveats stated with the numbers: the turn cap is not arm-neutral and the study's operator is a script.
  • Chrome's own webmcp-evals CLI: 24 of 24 smoke steps against the live URL with no model, and with GPT-5 the right tool and arguments in 28 of 28 cases, zero production writes in Triage, the smuggled log instruction named as an injection and refused. Lighthouse agentic browsing 4 of 4.

Every number, the command that regenerates it, and its caveat: docs/evals.md. The claim and how far it is defended: docs/thesis.md.

What we learned

Management side:

  • Applying good taste and judgment: Even highly capable models require constant steering on UX, product decisions, and even technicals, surprisingly. There were many times where I had to challenge the model on premises it was proposing, assumptions it was making, judgment calls that didn't make sense, etc. Cliche, but always trust your gut, it truly does not hurt at all to challenge your AI (either you're right, or you learn something).
  • Vibecoding a UX is tough: In retrospect, I feel like I should have spent more time upfront mocking up the UX. A lot of time in the development of the project was spent iterating on what the right UX felt like which was slow, cumbersome, and honestly even very frustrating at times. A clear mock up is a contract you can hold your agent to and reasonably expect it will converge on after some small number of iterations.
  • Testing along the way: Normal SWE principles, but never proceed to new features until the current state of the codebase has good test coverage and every net-new build can be empirically measured to not cause regressions. I think this kept the codebase in a coherent state throughout.
  • The power of leverage: AI is an amplifier. So every input matters - the quality of your prompts, the information you ask it to look for, etc. I've learned to let my impatience not distract from how much that can matter.

Technical side:

  • A description is the whole contract. Anything a caller could guess wrong has to be said, and an input that names nothing should fail loudly rather than return an empty page.
  • A write that changes nothing has to say why, in the tool result and on screen, or the agent's next move is a guess.
  • "Human in the loop" on a computer-use host is a convention, not a guarantee, until the spec gives pages a way to mark a control as human-only. Seven pieces of feedback to the spec are in docs/spec-feedback.md.
  • The measured study is a bonus, not the claim. It is reported with its confound intact rather than rounded off. Intellectual honesty.

What's next for Release Airlock

Ideally we move away from a simulation and actually attach it to a real system! The goal is to motivate the use-case and the potential impact on keeping systems and businesses healthy.

Built With

  • github-actions
  • playwright
  • typescript
  • vercel
  • vite
  • vitest
  • web-workers
  • webmcp
Share this project:

Updates

Submission history