SAVS: Visual Proof for WebMCP Agents

• 

  1 · Fresh R1 is blocked. The action succeeded. The rendered revision did not.

• 

  3 · Fresh R2 passes. Only a fresh audit of the repaired revision returns PASS.

• 

  2 · Four native WebMCP tools. State, action, audit, and repair are page-registered.

• 

  4 · Old evidence remains blocked. Visual evidence belongs to the exact revision it inspected.

Inspiration

WebMCP makes website actions explicit and reliable, but successful execution does not prove that the human-visible result is correct. An agent can invoke the right tool and still leave clipped text, a shifted baseline, an overflow, or another visual regression behind. People experience the rendered page; agents usually receive a structured action result. We built SAVS to connect those two meanings of success.

What it does

SAVS is an agent-native visual verification workbench for WebMCP. The workbench begins at approved reference revision R0. An agent invokes a valid compact-layout tool, which succeeds and creates immutable revision R1. R1 contains a small but visible two-CSS-pixel baseline defect in the metric strip.

The agent then audits R1. SAVS rerenders the approved reference and requested revision, compares their pixels, and returns a fresh BLOCK verdict with visual evidence. A constrained alignment repair creates successor revision R2. The earlier R1 result cannot approve R2 because its evidence is bound to R1 and is now stale. Only a fresh audit of R2 returns PASS.

Four native page tools expose this flow:

• get_visual_state reads the current immutable revision and audit state.
• apply_compact_layout performs the valid action that creates R1.
• run_visual_audit audits an explicitly requested revision and reports whether its evidence is fresh.
• apply_alignment_repair creates a successor revision using an allowlisted repair and an expected-revision precondition.

The tools are registered with document.modelContext.registerTool. They are semantic capabilities, not wrappers around screen coordinates. The visible controls use the same application endpoints, so the person and agent share one state, history, evidence surface, and set of verdicts.

How we built it

The reference app is self-contained. A Node.js server maintains isolated sessions, immutable revisions, idempotency receipts, and audit records. The browser renders the current revision and evidence while registering the four WebMCP tools. A controlled renderer produces approved-reference and candidate PNGs. The verifier checks exact pixel differences, protects a stable control region, limits accepted changes to the intended target, and binds every audit to its requested revision.

The same R0 → R1 → R2 behavior is exercised through native WebMCP discovery and invocation in Google Chrome and through an independent verifier against the public Render deployment. The live app requires no account or credentials. The public repository contains the standalone source, tests, instructions, MIT license, and Challenge-period provenance.

Challenges we ran into

The hardest part was not detecting that pixels changed. It was making the result trustworthy across time. An audit that was valid for R1 must not become approval for R2, and a repair must not overwrite the history that explains the failure. That led us to immutable revisions, freshness-aware verdicts, expected-revision checks, and idempotent action receipts.

The demo also exposed a more human problem: a two-pixel baseline defect can be obvious to a viewer while looking harmless in code or DOM geometry. We made the rendered pixels—not styling intent—the final evidence boundary.

Accomplishments that we are proud of

• Four real page-registered WebMCP tools form one coherent workflow.
• A technically successful action produces a visible defect and a structured fresh BLOCK verdict.
• Stale evidence remains blocked after the page advances to a new revision.
• A constrained repair creates R2 without erasing R1.
• A fresh R2 audit returns PASS with zero changed pixels.
• The same behavioral chain has passed against the public Render origin.
• The accepted 2:37 demo uses real native WebMCP calls, audible narration, and burned-in English captions.

What we learned

Structured tools solve action ambiguity, but agents also need structured evidence about outcomes. Visual approval is not a timeless property of a page; it belongs to a particular rendered revision. That small change in the contract makes failure, repair, and approval understandable to both the agent and the person supervising it.

What's next

This Challenge workbench proves the smallest complete loop. The longer-term direction is to generalize the same revision-bound visual contract across real applications, responsive states, and richer human-approved visual intent while keeping the evidence inspectable by both people and agents.

Challenge-period provenance

SAVS began as pre-existing visual-verification research. During the Challenge period, we built this self-contained WebMCP reference implementation: a new page-registered four-tool interface, immutable revision workflow, freshness-bound visual verdicts, constrained repair path, native WebMCP browser verification, public deployment, and Challenge demo. This repository does not import the private SAVS research runtime.

Built With

• chrome
• chromium
• css
• html
• javascript
• node.js
• playwright
• png
• render
• webmcp