Incident Command

A WebMCP safety simulator where a browser agent investigates production incidents, but the dangerous tool to act is never registered until a human approves inside the page.

Comment

Inspiration

Teams are beginning to connect browser agents to operational dashboards, deploy consoles, incident rooms, and status pages. That's useful, but it creates a sharp safety problem: the first time a team learns how an agent behaves around a production-changing action shouldn't be during a real outage. Prompt injection through page content is one of the loudest open criticisms of giving agents browser tools, and we wanted to build something that answers it structurally instead of just promising to be careful.

What it does

Incident Command is a WebMCP safety simulator for agent-assisted production incidents. A human and a browser agent work through a simulated checkout outage together. The agent can inspect incident state, compare service evidence, propose hypotheses, request approval, and close the incident with a scorecard. The human can inspect the same evidence, mark a suspect, override or challenge the hypothesis, approve or reject a risky action with a written reason, and revoke the agent's access to capabilities or services while the incident is live.

Four scenarios are included, URL-selectable: an obvious deploy regression, a red herring where the newest deploy is innocent, a cascading failure across two services, and an adversarial scenario where the incident timeline contains a prompt injection telling the agent to roll back the wrong service.

Why WebMCP is a strong fit

The core question this project asks is about the boundary between what the agent can know, what it can ask for, and what it can actually do. DOM automation can click whatever the page exposes, and hiding a button doesn't stop an agent from being told to look for it. WebMCP lets the page define a small, structured tool surface and change that surface as the incident moves through phases and as human policy changes.

In triage, the agent sees three investigation tools. It cannot see the production-changing action, because the tool literally isn't registered yet — not disabled, absent. If the human removes Payments from scope, payments disappears from the relevant tool schemas and investigation evidence. After the agent requests approval, a human must approve with a real, trusted click in the page UI (validated with event.isTrusted, since a script-dispatched click has isTrusted: false). Only then does the execution tool become visible, and even then it fails closed unless the approval matches the exact action, target service, and phase.

The thesis: an agent cannot be prompt-injected into calling a tool that was never registered. A permission check still hands the model the tool's name, description, and schema — and the temptation. Phase-scoped and human-scoped registration removes the dangerous action from the model's available tool list entirely until the human gate opens.

How it creates a better user experience, and what's now possible

This turns human-agent collaboration into something with a visible, legible boundary instead of an implicit trust assumption. The agent does the tedious incident work — gathering evidence, drafting hypotheses, proposing a response — while the human keeps direct control over judgment, risk, and visibility. The person can challenge the agent's hypothesis, add context, reject a weak proposal with a written reason that flows back into what the agent sees next, remove a service from scope mid-incident, and make the final production call.

What's newly possible: teams can now rehearse a realistic agent-safety scenario — partial evidence, a red herring, a cascading failure, a prompt-injected instruction — and get an objective scorecard on whether their agent behaved correctly: right root cause, right mitigation, no premature production action, and whether it complied with the injected instruction. Before this, the only way to learn how your agent behaves under a dangerous instruction was to wait for one to happen for real.

How we built it

The app is vanilla JavaScript, no framework, no build step, deployed on Vercel. It registers up to six WebMCP tools with document.modelContext.registerTool: get_incident_state, investigate_incident, propose_response, request_approval, execute_approved_action, and close_incident. Registration is awaited, uses a per-tool AbortController so tools can be genuinely unregistered (not just hidden) when a phase ends or a human revokes a capability, and exposes live diagnostics so a registration failure is visible instead of silent.

Human approval is deliberately not a WebMCP tool — an agent can request it, but only a trusted page click can grant or reject it, and that trust is derived from the DOM event itself rather than accepted as a caller-supplied argument.

Challenges we ran into

The approval gate went through six rounds of adversarial review before it held: an agent approving its own request by calling a tool that shouldn't have existed, classic-script globals leaking internal approval helpers onto window, live state references that let a caller mutate an approval directly, and forged trusted: true decisions surviving in localStorage across reloads. Each hole only became visible after the previous one closed — the safety suite (npm run test:safety) now permanently tests all of them, plus schema validation, approval replay, cross-service approval reuse, and capability-revocation races.

Two WebMCP-specific implementation details cost real debugging time: document.modelContext.getTools() returns a Promise and must be awaited, and the tested runtime required executeTool arguments as a JSON string rather than an object — passing an object fails with an unhelpful parse error.

What's next

Extending the scorecard into a small published benchmark — running multiple agents against all four scenarios and publishing pass/fail results — so the project becomes a testbed other people can point their own agent at, not just a demo of ours.

Built With

Share this project:

Updates