ExactDelta

A pre-release effect gate that proves a WebMCP write changed exactly what the human authorized—and nothing else.

Comment

Inspiration

WebMCP gives browser agents a structured way to act on websites. That makes tool selection and argument validation easier to inspect—but a developer or QA engineer deciding whether a write tool can ship still needs evidence about its actual state transition. A handler can accept the right target, return success, and still mutate an unselected record.

The WebMCP security discussion explicitly says a tool's declared intent is not guaranteed to match its behavior and that an agent cannot confirm the real effect from the description alone. Chrome's own Evals guidance recommends classic deterministic tests for UI updates and intentional side effects. ExactDelta makes that effect boundary generated, inspectable, and reusable instead of claiming to replace call evals or application tests.

We wanted one failure to be understandable without background knowledge: the agent did everything right; the result was still wrong.

Approval proves what an agent may attempt. ExactDelta proves what the application actually changed. Schemas validate arguments, approval gates authorize requests, and traces record calls. Those controls are necessary, but none establishes that the handler changed only the state the human authorized. ExactDelta begins at that missing post-action boundary.

The target user is a developer or QA engineer shipping browser-agent writes into commerce, support, administration, or access-control workflows, where an accepted action can still alter the wrong order, permission, or customer record. ExactDelta turns that failure into a visible release decision and an executable repair regression rather than leaving the team to reconstruct it from a trace.

What it does

In an owned staging environment, the human selects one target in the application UI. Before the action, ExactDelta turns that visible intent and the pre-action snapshot into an Effect Contract:

  • the selected record must make the declared transition;
  • every unselected record must remain unchanged;
  • entity identity and count must remain stable.

The browser agent then directly invokes the registered native WebMCP tool. The invocation itself enters ExactDelta's gate. ExactDelta does not trust success: true; it snapshots application-owned post-action state independently of that payload and separates required changes from unexpected changes. The native response preserves the action result and adds both an independent effectGate verdict and the executable exactdelta.regression.v1 artifact. A checked-in runner loads that exact JSON, rejects schema or contract drift before the write, replays the recorded arguments, and requires the repaired implementation to return ACTION_PROVEN with identical regression identity.

The demo uses two fictional workflows—order cancellation and permission change—with the same verification core.

That core is not tied to the demo store. The repository builds a typed, zero-runtime-dependency ESM SDK whose runEffectGate() accepts an application-owned intent/snapshot adapter and action binding. A package audit type-checks a NodeNext consumer, packs and installs the tarball into a fresh project, then makes a third support-ticket adapter detect a collateral write and replay the identical artifact to ACTION_PROVEN after repair; registry publication is not claimed.

Why this is a WebMCP use case

ExactDelta wraps the page-exposed action boundary that WebMCP creates. It registers only the tool relevant to the visible workflow with document.modelContext.registerTool(), disposes it when context changes, discovers it with getTools(), and invokes it with executeTool(). Exact schema enums bind the target and requested value to visible intent, while same-origin exposure narrows who can invoke it. A call from the browser client enters the verifier automatically; the deterministic Run control invokes the same native path as a stable judge fallback.

Without WebMCP, ExactDelta would be a conventional application-specific regression test. With WebMCP, it discovers and executes the same structured write boundary exposed to agents, so the human's visible intent, the agent-facing page action, and the application's resulting state can be inspected as one release proof chain.

Better human-agent UX

The QA operator does not write a record-specific assertion or decode an agent trace. The visible selected target defines the authorized boundary; the browser agent discovers and invokes the corresponding tool. ExactDelta generates the effect boundary, shows the exact call, separates the tool result from observed state, highlights collateral changes, makes the release decision visible, and preserves a regression that developers can rerun after a fix.

What the human and agent can do together

Before this flow, a QA engineer could inspect an agent trace and separately author application-specific state assertions, but the browser agent's successful tool response did not itself establish which records were allowed to change. ExactDelta joins those responsibilities at the page boundary: the human's visible selection declares the permitted effect, the agent invokes the same native WebMCP action it was given, the application independently observes the resulting state, and the agent receives both the separate release-gate verdict and its executable regression artifact. The developer can then rerun the exact returned contract after repair instead of reconstructing the failed case from a trace.

How we built it

  • React 19, TypeScript, and Vite
  • OpenAI Codex as the primary development environment; GPT-5.6 for architecture review, test design, and adversarial submission review
  • Native imperative WebMCP registration and execution
  • Read-only native input-mode detection for Chrome and in-app-browser executeTool() dialects, with exactly one application write
  • Context-matched tool lifecycle, exact visible-intent schemas, and same-origin exposure
  • Typed Effect Contract generation from explicit selection and pre-state
  • Distribution-ready typed ESM SDK with a public application-owned adapter API
  • Deterministic snapshot diff for required, unexpected, and invariant changes
  • Visible pre-release gate state and downloadable exactdelta.regression.v1 CI artifact
  • JSON-driven CI runner with schema, intent, arguments, contract, and regression-identity enforcement
  • Vitest, Playwright native-Chrome E2E, axe-core WCAG A/AA checks, and deterministic build/runtime gates
  • Controlled comparison using the official webmcp-evals 0.0.4 matcher plus manual Playwright state assertions

Controlled comparison

Across two deterministic native WebMCP workflows, the official Evals matcher passed 2/2 correct calls, rejected 2/2 wrong-argument controls, and still left collateral defects in 2/2 resulting states. Adding four concrete expected-state assertions in Playwright caught both defects and passed unchanged after repair. ExactDelta caught both and passed both identical retained regressions using two reusable action bindings and zero per-record expected-state assertions in the scenario definitions.

This comparison measures detection coverage only. ExactDelta still requires an application-owned state adapter and an action binding for each action class.

Challenges

The hardest design constraint was avoiding a dressed-up Playwright wrapper. The Effect Contract had to be generated from visible intent and pre-state, remain inspectable, work across two action classes, and demonstrate a measurable boundary from expected-call matching without exaggerating the comparison.

We also kept native and fallback evidence separate: the UI labels harness mode, while the browser suite requires a real Chrome WebMCP API. Chrome and ChatGPT's in-app browser currently accept different executeTool() input forms, so we resolve that difference with a read-only startup probe instead of retrying a state-changing call.

Accomplishments

  • A real browser-client WebMCP invocation that automatically enters the effect gate
  • A 20-second silent proof of correct call / failed effect
  • An explicit product decision: release blocked on collateral change, passed after identical repair regression
  • One verification core for order and permission workflows
  • The same downloaded JSON artifact can be loaded and executed as a CI release gate
  • A reproducible, source-visible Evals + Playwright comparison
  • A fresh-project packed-install proof using a third workflow
  • Zero automated WCAG A/AA violations in initial and blocked states, with bounded build and cold-browser performance gates
  • No credentials, external writes, or private data

What we learned

An invocation trace answers “what did the agent ask the page to do?” An Effect Contract answers a different question: “what state was allowed to change?” Both are useful, and neither should be presented as a substitute for the other. Public tools such as webmcpify already perform competent real-browser result and declared UI-delta verification; ExactDelta's narrower contribution is generating the exact allowed delta and all-unselected-unchanged obligations from the current visible target and pre-state.

What's next

A production-oriented deployment would add authenticated server-state adapters, bounded polling for delayed effects, normalization for volatile metadata, and authorization-aware contract scopes. Those extensions are intentionally outside this submission; the current SDK proves the integration boundary without claiming production adoption.

Built With

  • codex
  • github-actions
  • gpt-5.6
  • open-source
  • playwright
  • react
  • typescript
  • vercel
  • vite
  • vitest
  • webmcp
Share this project:

Updates

Submission history