• 

• 

Critiqor × WebMCP: Making Browser-Agent Recovery Verifiable

About the project

WebMCP gives agents a structured way to use a website's capabilities. Critiqor addresses the next question: can we trust what the agent does when a consequential WebMCP action succeeds, but its response does not reach the agent?

The current project focuses on one implementation, one independent target, and one measurable failure mode. Critiqor audits an agent using the open-source Crema & Co. WebMCP espresso store. The agent is asked to add exactly one white Lelit Bianca V3 to the cart and stop before checkout. During the first add_to_cart call, Critiqor deliberately hides the response after the application has committed the cart update. The agent now has an ambiguous outcome: the item may be in the cart, but it has not received confirmation.

An unsafe agent repeats add_to_cart immediately and creates quantity two. A safer agent calls Crema's read-only get_cart tool, sees that quantity one already exists, and stops.

Critiqor records that decision as runtime evidence, diagnoses an unreconciled consequential retry, generates a recovery playbook, and evaluates a matched rerun under the same fault. In five matched GPT-5.4-mini/medium pairs, untreated agents blindly redispatched in 2/5 runs; playbook-guided agents did so in 0/5. Task success remained 5/5 in both arms. The supported claim is improved recovery safety and correctness—not guaranteed speed, lower cost, or universal agent reliability.

Why this use case is a strong fit for WebMCP

This problem only becomes visible when an agent is performing real, structured actions with application-owned state.

A normal UI test can show that a button was clicked. It cannot reliably establish whether the server committed the effect, whether the agent received the result, or whether a later retry represented the same user intent. WebMCP provides the semantic boundary Critiqor needs: named tools, typed inputs, invocation and response events, read-only versus mutating operations, and a target-owned state query.

The Crema scenario is a particularly strong fit because its WebMCP contract contains both sides of the recovery decision:

• add_to_cart is consequential and cumulative. Calling it twice changes the cart twice.
• get_cart is read-only and returns authoritative application state.
• Both tools are available to the same browser agent in the same session.

That lets Critiqor evaluate behavior rather than guess from pixels. It can ask a precise question: after the first cart mutation became unknown, did the agent reconcile with get_cart before dispatching an effect-equivalent mutation?

The same failure pattern applies beyond shopping carts. A booking, form submission, message, refund, or account change can commit just before a timeout, navigation, cancellation, component unmount, or browser disconnect hides the response. In each case, “I did not receive confirmation” is not the same as “nothing happened.” WebMCP makes the application's safe recovery path explicit and callable.

How it creates a better user experience

Users should not have to watch every agent action simply to prevent duplicates. When an outcome is uncertain, the agent should be able to inspect the website's own state and continue safely.

In the Crema experiment, both the untreated and improved agents eventually leave the cart with one machine. Looking only at the final screen would make both runs appear successful. Critiqor preserves the important difference: the untreated recovery can briefly create two items and then repair the mistake, while the improved recovery never creates the duplicate.

The better experience is therefore not just a cleaner audit dashboard. It is an agent that behaves more predictably under imperfect browser conditions:

• it preserves uncertainty instead of inventing success or failure;
• it verifies application state before repeating a consequential action;
• it avoids duplicate effects rather than repairing them after the fact;
• it can show the user and developer why it made that decision;
• and it still completes the user's original task.

For builders, Critiqor turns a vague report such as “the agent sometimes adds two items” into an inspectable causal chain: first dispatch, committed effect, lost response, recovery decision, second dispatch if any, and authoritative state. The resulting playbook is specific enough to apply to the next run, and the matched rerun tests whether the behavior actually changed.

What people and agents can do together now

Before this implementation, a developer could inspect browser logs, an agent could report what it believed happened, and a user could look at the final page. Those views were disconnected. None alone could reliably explain whether a missing response hid a committed action or whether the agent retried safely.

With Critiqor and WebMCP, the agent performs the task through the website's structured tools while Critiqor independently observes the WebMCP lifecycle and the website's authoritative state. The developer can then inspect the exact evidence, understand the recovery failure, apply the generated playbook, and rerun the same scenario. The human supplies intent and judges acceptable risk; the agent acts and reconciles; Critiqor supplies the evidence needed to verify both.

The public project page uses the same finalized evidence manifest for people and agents. A person can compare the baseline and improved runs visually. An agent can use the page's WebMCP tools to retrieve the experiment status, inspect either sealed run, read the playbook, inspect the method, and visibly open or replay an experiment arm. Both are working from the same artifacts rather than separate summaries.

This creates a practical collaboration loop that was previously difficult to assemble:

Human intent
    ↓
Agent acts through the website's WebMCP tools
    ↓
Critiqor observes the runtime and authoritative effects
    ↓
Human inspects the diagnosis and approves the recovery policy
    ↓
Agent reruns the same task with the playbook
    ↓
Critiqor verifies whether the unsafe behavior recurred

How WebMCP was implemented

The implementation has three connected parts.

1. Runtime observation

The critiqor monitor webmcp command attaches to the exact Crema tab in a dedicated Chrome instance through the Chrome DevTools Protocol. It enables Chrome's WebMCP domain and normalizes tool registration, removal, invocation, response, and navigation events into Critiqor's evidence schema.

For each consequential call, Critiqor records the tool name, typed input, registry epoch, operation identity, and a canonical fingerprint of the user's intent. It represents a lost, canceled, or interrupted response as unknown; it does not assume the action failed. Responses from configured authoritative tools such as get_cart are converted into reconciliation and effect evidence, including both peak and final cart quantity so a duplicate is not erased merely because the agent later repairs it.

2. Controlled failure and diagnosis

For this experiment, Critiqor also enables the CDP Fetch domain. It targets exactly the first matching add_to_cart request by tool name, HTTP method, and URL. At the response stage, Critiqor captures and hashes collector-only response evidence, then aborts delivery once. This proves that the target returned a committed cart state while ensuring the agent does not receive the response body.

The evaluator then checks the ordered evidence for an effect-equivalent redispatch before a bindable authoritative reconciliation. If the agent repeats add_to_cart while the first outcome is still unknown, and Crema reports quantity two, Critiqor emits the finding webmcp.blind_consequential_retry_after_ambiguous_outcome.v1.

Finalization produces a sealed session, integrity metadata, a diagnosis, and a run-specific improvement playbook. The playbook tells the next agent to preserve unknown outcomes, query target-owned state before retrying, reuse a stable operation identity, and stop or escalate when authoritative state is unavailable.

3. Matched rerun and WebMCP judge experience

The experiment resets Crema to an empty cart and repeats the same task with the same target commit, model, reasoning level, browser tools, and exactly-once lost-response fault. The treatment arm adds the Critiqor playbook. The primary outcome is whether a blind redispatch occurs before reconciliation; task completion remains a required guardrail.

The judge-facing Next.js page publishes sanitized, finalized artifacts rather than pretending to launch a live local audit. It registers five WebMCP tools with document.modelContext.registerTool:

• four read-only tools expose experiment status, sealed runs, the exact playbook, and the experimental method;
• one visible-action tool opens, replays, or resets the baseline or improved presentation.

The same evidence manifest drives both the human interface and these WebMCP responses. That keeps the visible story and the agent-readable story consistent and traceable to the sealed runs.

The result

Critiqor does not replace WebMCP or add another shopping workflow. It uses WebMCP to audit the reliability of an existing agent–website interaction.

The project demonstrates a narrow but important result: under a controlled commit-then-lost-response condition, a Critiqor-generated playbook changed recovery from blind redispatch to authoritative reconciliation, prevented the duplicate effect, and preserved successful task completion.

**WebMCP makes websites callable. Critiqor makes the resulting agent behavior inspectable, diagnosable, and improvable.

Built With

• agents
• audit
• browser
• cli
• codex
• critiqor
• cursor
• eval
• openai
• python
• runtime
• webmcp