Signett

What is Signett

Signett is a reliability and evaluation layer for production WebMCP applications.

In our live Saleor demo, an agent discovers catalog and checkout tools, finds a product, requests human approval, and submits a purchase request. What makes it special is that the storefront updates only after the purchase is verifiably complete, as provided by Signett. We also simulate a browser response loss after the mutation commits and show how Signett recovers the authoritative result instead of creating a duplicate.

What it does

WebMCP gives agents a structured way to use website capabilities. Signett adds the execution semantics and developer workflow needed to operate those capabilities reliably.

The Signett SDK accomplishes that through several core features:

• JSON Schema validation before application code runs
• trusted application context and application-owned authorization
• human confirmation for consequential effects
• idempotency and replay coordination
• operation journaling and ambiguous-failure recovery
• authoritative postcondition verification
• cancellation and structured repair guidance
• privacy-conscious OpenTelemetry traces
• React bindings for verified UI updates
• deterministic testing without a model or browser

Simple read tools remain lightweight. Consequential tools can use the complete guarded lifecycle. Signett does not replace WebMCP, authentication, backend transaction guarantees, or application business logic. The website continues to own identity, policy, state, consent UI, and every backend effect. Signett creates a structured boundary around those existing capabilities.

The wider project includes the TypeScript SDK, a Chrome side-panel agent, a headless evaluation CLI, a local Inspector, OpenTelemetry export, reference applications, reproducible benchmarks, and reviewed evidence.

Inspiration - Why Do We Need This?

I am a strong believer that the web will be largely agentic in a few years. While we have seen a lot of promise with browser-use agents, what bothered me the most was that these harnesses were trying to operate on websites built for humans, clicking buttons, while they should be using tools/MCPs.

Then, I discovered WebMCP and was instantly excited. However, I quickly found that while the protocol is extremely valuable, a real developer trying to make their websites agentic doesn't have an ecosystem built around it. Specifically, I saw several problems:

What happens when an agent retries a purchase? What if the backend commits an effect but the response disappears? How does the user retain control over consequential actions? When should the UI update? How does a developer test the interface repeatedly, inspect traces, or detect a regression?

These are familiar distributed-systems problems, but agentic websites make them more common and more consequential.

Signett was built to answer a specific question: How do we move from “an agent can call this tool” to “an agent can perform this action safely, predictably, and verifiably”? That idea became Signett.

How we built it

Signett is a TypeScript monorepo with four primary layers.

Signett SDK

createSignett() creates a WebMCP-facing application surface. signett.expose() compiles the tool’s schema and registers it through the native document.modelContext.registerTool() API. Each tool defines its agent-facing contract and existing application function. Optional hooks add context, authorization, confirmation, idempotency, recovery, verification, output budgets, and lifecycle observation.

Signett Chrome Agent

The Chrome side-panel agent discovers and invokes only the WebMCP tools exposed by the active page. It deliberately has no DOM or screenshot fallback, so missing or poorly designed capabilities remain visible to developers.

Signett Eval

The CLI opens the real application in headless Chrome, discovers its live WebMCP inventory, runs saved tasks, and records bounded evidence. Applications can provide their own reset, snapshot, and grading hooks. This means success can be determined from the database or domain model rather than from the agent’s final response. signett check compares evaluation results against a reviewed baseline and can detect regressions in task success, safety, capability discovery, tool selection, argument validity, latency, token usage, and trial coverage.

Observability and benchmarks

Signett emits lifecycle events covering validation, authorization, confirmation, execution, replay or recovery, output measurement, and verification.

These can power an in-page Inspector or be exported as OTLP/HTTP JSON to systems such as Jaeger. Inputs, outputs, application context, error messages, and stack traces are excluded by default.

The repository also includes deterministic safety tests, real-agent evaluations, reference integrations, benchmark methodology, and reviewed evidence.

Challenges we ran into

One of the hardest problems was defining correct behavior after an ambiguous failure: a server may successfully commit an order or payment while the response is lost. Blindly retrying could duplicate the action, but blindly reporting failure would also be incorrect. Signett combines durable idempotency records, operation journals, recovery logic, and authoritative state verification to handle this case explicitly.

Authorization and idempotency also interact in subtle ways. Returning a cached result must never bypass a current permission check. Signett therefore reevaluates authorization before every idempotency lookup, including replays.

Another challenge was making observability useful without turning it into a privacy risk. Signett records lifecycle metadata and timing information by default, while avoiding tool inputs, outputs, application context, and stack traces unless the application explicitly chooses otherwise.

Building high-quality benchmarks to hill-climb on was hard, but I think we got a lot of good-quality evaluations. Check out the benchmarks folder in the open-source repository.

Finally, WebMCP is still experimental and evolving. I had to design useful abstractions without hiding the native API or prematurely inventing a competing protocol.

Accomplishments that we're proud of

Signett became more than a wrapper around tool registration. It now supports an end-to-end developer loop:

1. Expose a native WebMCP capability.
2. Inspect and invoke it in Chrome.
3. Test its execution invariants deterministically.
4. Exercise it with a real model in headless Chrome.
5. Grade the result against the authoritative application state.
6. Inspect its lifecycle through OpenTelemetry.
7. Compare the run with a reviewed baseline.

The same live WebMCP tools power the browser agent, the application UI, deterministic tests, and headless evaluations. There is no parallel demo-only tool catalog.

We also validated the model against larger applications:

• Saleor: checkout, approval, lost-response recovery, replay, and authoritative verification
• Cal.diy: a real agent completed a booking after an injected lost response, with exactly one booking independently observed in Postgres
• an authenticated payment application used for deterministic safety and repeated agent-effectiveness studies

What we learned

The biggest lesson was that defining a tool schema is not the same as creating a trustworthy execution boundary.

Agentic actions amplify several familiar distributed-systems problems. Retries, concurrency, authorization changes, cancellations, and lost responses become especially important when an agent can make purchases, send payments, or change persistent state.

I also learned that the agent’s response is not proof that a task succeeded. Consequential actions need to be verified against the authoritative application state. The same principle applies to evaluation: the application’s database or domain model should grade success, not the agent transcript.

Another important lesson was that deterministic testing and probabilistic agent evaluation serve different purposes. Deterministic tests prove safety invariants. Repeated agent trials reveal whether tools are discoverable, understandable, and effective.

Most importantly, I learned that reliability features must remain composable. A read-only search tool should be easy to expose, while a payment or order mutation should be able to opt into stronger controls.

Benchmarks and Results

Signett's benchmarks grade consequential tasks against authoritative application state, not the agent's final answer or the tool's own response. Safe success means that the requested goal was reached without a forbidden or duplicate effect.

Benchmark metric	Baseline	Signett	Improvement
Safe recovery success across 42 real-agent runs	Raw WebMCP: 2/21 (9.5%)	21/21 (100%)	10.5x success rate
Unsafe retries during recovery	Raw WebMCP: 12/21	0/21	All observed unsafe retries eliminated
Median recovery time	Raw WebMCP: 20.9 s	14.1 s	32.5% faster
Median recovery tokens	Raw WebMCP: 140,980	114,320	18.9% fewer
Safe task success in the 60-run UI/WebMCP study	UI only: 10/20 (50%)	15/20 (75%)	+25 percentage points
Deterministic fault scenarios passed	Raw WebMCP: 3/7	6/7	2x as many passed
Duplicate effects under faults	Raw WebMCP: 3	0	All observed duplicates eliminated
Bespoke adapter code	Hand-built controls: 29 SLOC	17 SLOC	41.4% less code

The real-agent recovery benchmark used identical tools in both conditions and was graded from database state. The deterministic suite covered lost responses, reloads, concurrent calls, stale state, invalid arguments, and concurrent writes. Signett still does not prevent every application-level race; the remaining lost-update case is detected and reported rather than hidden.

UI-to-tool gains demonstrate the value of WebMCP, while raw-to-guarded safety and recovery gains demonstrate the value added by Signett. Full methodology and reviewed evidence are available in the benchmark reports.

What's next for Signett

The next priorities are:

• propagating one trace context across an entire agent session and all page-owned tool calls
• expanding randomized execution-safety testing
• validating Signett on more held-out applications and with external developers
• inviting independent reproduction of the published benchmarks
• adding prerequisite and dependency metadata for tools that must not run concurrently

The goal is to make production WebMCP development feel like a complete engineering discipline: explicit capabilities, safe execution, observable outcomes, and evidence that the interface continues to work.

Built With

• chrome
• codex
• typescript