EqualTrace

The agent succeeded. The safety contract failed.

Comment

Problem

WebMCP lets an agent call structured tools directly from a live website. That can be dramatically more reliable than guessing through a visual interface—but it introduces a new release risk. An agent can reach the correct final state while silently bypassing disclosures, exact consent, completion feedback, reversibility, or recovery protections that constrained a person performing the same consequential action.

Outcome-only testing calls that a success. EqualTrace calls it a protection failure.

Solution

EqualTrace is a runtime protection-parity workbench for agent-operable websites. It runs one deterministic fictional account-deletion action through three genuine routes:

  1. a visual pointer journey;
  2. a keyboard/assistive journey;
  3. a native WebMCP agent invocation.

All three routes use the same scenario, seed, domain commands, requested outcome, and semantic trace model. EqualTrace compares outcome, disclosure, consent, feedback, reversibility, and recovery in their required order. The baseline intentionally reaches deleted on all three routes, but fails at the first missing agent checkpoint: disclosure.consequences.

The agent can then stage one bounded repair, but it cannot approve or apply it. A person must inspect and approve the exact action, checkpoint, digest, seed, and expiry through the visible interface. Only then does equaltrace_apply_approved_repair appear on the native WebMCP surface. It is bound to that authority, executes once, and disappears immediately after use or any invalidating transition.

EqualTrace clears the baseline traces, reruns all three routes, and issues a deterministic SHA-256 parity receipt only when the same outcome and all six protection groups pass on fresh evidence.

Why This Matters

The central failure and the central proof exist on the real capability surface. EqualTrace does not merely display tool metadata. Native discovery and invocation update the same visible state used by people, while dynamic registration makes authority observable: four stable tools exist before approval, a fifth exact tool appears after human approval, and it disappears after one attempt.

Without WebMCP, EqualTrace could not demonstrate the difference between interface protections and agent-call protections, nor prove that authorization changes the agent's available capabilities rather than a decorative status label.

Better human-agent collaboration

Before EqualTrace, a team could separately test a UI, accessibility, and an API call yet still miss that the agent path skipped a protection. EqualTrace lets the agent diagnose and prepare a narrow repair, keeps the consequential decision with a person, and lets the agent execute only the approved change. Both then share a portable receipt showing exactly what passed.

This pattern can extend to account management, privacy controls, commerce, healthcare, finance, and other consequential actions where a correct outcome is not enough evidence of a safe path.

Architecture

EqualTrace is a static React and strict TypeScript application with no backend, database, account, secret, or model API. document.modelContext.registerTool exposes four constant, bounded stable tools. An AbortSignal-bound lifecycle manager dynamically registers the single-use repair tool only after exact visible approval.

Every route executes isolated run contexts from one immutable fixture and shared domain engine. Deterministic trace IDs feed a pure fail-closed comparator. Canonical JSON and Web Crypto produce the repair digest and receipt identity. Automated tests use a separate injected fake port and are always labelled simulated; native claims come only from a supported client's browser-provided WebMCP surface.

The one-command gate runs formatting, lint, strict type checking, 66 unit/integration/component tests with authority coverage thresholds, production build, 12 Playwright journeys, axe checks, receipt verification, and native-evidence manifest verification. The exact public release also completed five consecutive native golden runs from fresh resets with the same receipt.

What We Learned

The hardest part was not registering a tool; it was defining what equivalent protection means across fundamentally different interaction modes. The implementation needed semantic checkpoints rather than screen similarity, fail-closed evidence rules rather than optimistic status flags, and authority bound to exact state rather than a claimed actor identity.

We also learned to keep semantic receipt identity separate from volatile browser and time metadata, while preserving native environment evidence in a separate auditable manifest.

How We Used AI

The demonstrated agent operates the live public page through native WebMCP tools. OpenAI Codex assisted with product planning, implementation, test design, adversarial review, accessibility checks, debugging, native browser validation, video production, and submission preparation. The running product calls no model API: its scenario, comparator, repair authority, and receipts are deterministic and reproducible without an external service.

How We Used Codex

Codex helped turn the product thesis into a strict TypeScript implementation, build the shared three-route domain engine, test fail-closed authority boundaries, inspect the real browser-provided WebMCP surface, and produce auditable release evidence. Throughout the project, simulated Playwright and Vitest evidence remained explicitly separate from native supported-client observations.

Key Features

  • One deterministic consequential action exercised through visual, keyboard/assistive, and native WebMCP routes.
  • Six ordered protection groups: outcome, disclosure, consent, feedback, reversibility, and recovery.
  • First-divergence diagnosis even when every route reaches the same final outcome.
  • Exact visible human approval before a consequential repair capability can exist.
  • Observable native capability lifecycle: four stable tools, one temporary approved tool, then four again (4 → 5 → 4).
  • Fresh repaired rerun with all three routes at 6/6 and a deterministic SHA-256 receipt.
  • Responsive static deployment with no backend, account, secret, external API, or model dependency.

Known Limitations

The challenge release deliberately proves one fictional account-deletion scenario. It is not a universal safety certification, general WCAG scanner, or external-site crawler. Public native release evidence covers the named Codex In-app Browser version; other WebMCP clients require their own validation.

After the challenge, the same model could support additional invariant packs, fixture authoring, CI regression runs against deployed WebMCP sites, and team evidence retention.

Built With

Share this project:

Updates

Submission history