Release Evidence Room

The agent builds the release evidence case. The human makes the call.

Comment

Start here

Inspiration

Release decisions are often fragmented across dashboards, test systems, network traces, and chat. A release can report 18 of 18 tests passing while still lacking evidence that a response-loss retry is safe. Scraping screens gives an agent context, but not a stable contract for evidence provenance, bounded access, safe mutation, or human approval. I built Release Evidence Room to turn the review page itself into a typed collaboration surface for the agent and the release owner.

What it does

Release Evidence Room starts with a public fictional checkout sandbox. The first operation is accepted but its response is lost; retrying the same intent with a new idempotency key creates a second modeled operation. The browser hands that exact synthetic session to the Release Evidence Room.

The page exposes five narrow WebMCP tools. An agent can read the release snapshot, query only page-owned sanitized evidence, propose a missing regression test, run an approved verification, and propose a verification-linked READY or HOLD decision. The human must approve the test before verification can run, and only the human can confirm the final release decision.

The approved verifier supports an exact targeted replay and a deterministic seeded monkey run. In the intentionally risky fixture, both reproduce two modeled side effects and support a HOLD recommendation. This demonstrates a release risk; it does not claim that a real customer was charged twice.

How I built it

The application is a two-route React and TypeScript project built with Vite. The Release Evidence Room registers exactly five imperative tools through document.modelContext.registerTool: get_release_snapshot, query_network_evidence, propose_test_case, run_approved_verification, and propose_release_decision.

Every tool has a strict JSON schema with additionalProperties: false and a separate runtime parser. The UI and the tool handlers use the same pure release-domain transitions. Versioned localStorage adapters validate both structure and cross-field consistency before restoring a checkout session or release state.

State-changing calls require the current expectedStateVersion and a reusable clientRequestId. Stale calls fail with state_conflict, while exact retries replay their original result without creating duplicate work. Verification runs only inside a deterministic page-owned sandbox and cannot fetch a URL, execute arbitrary code, access a live payment system, approve a proposal, deploy a release, or confirm the final decision.

Vitest covers the state machine, evidence handoff, domain rules, persistence, components, verifier, and WebMCP handlers. Playwright covers the complete checkout-to-HOLD path. The application is deployed on Vercel.

Challenges I ran into

The main challenge was making the agent useful without giving it release authority. Approval and final confirmation therefore are not merely prompt instructions: no WebMCP tool exists for either action.

A second challenge was demonstrating a realistic retry risk without exposing company data, raw packets, credentials, customer details, or a real payment backend. I solved this with a deterministic fictional checkout model and opaque, page-owned evidence references.

The third challenge was making agent retries and concurrent-looking state changes safe. Optimistic state versions reject stale mutations, idempotent request IDs prevent duplicate proposals and verification runs, and deterministic seeds make bounded monkey verification reproducible.

Accomplishments that I am proud of

The public application completes a two-route checkout-to-release workflow without requiring an account, card, credential, backend, or real payment. It exposes exactly five bounded WebMCP tools and keeps human approval and the final release decision outside the agent interface.

A clean native run in ChatGPT's in-app browser completed the checkout handoff, tool discovery, approval guard, targeted replay, seeded monkey verification, idempotent replay, stale-state rejection, READY guard, human-confirmed HOLD, and reload persistence.

The repository reports 88 passing automated tests. Its production build, Playwright integration path, and high-severity dependency audit also pass. All submitted evidence is fictional, synthetic, and privacy-safe.

What I learned

WebMCP is most valuable as a narrow domain contract rather than a collection of automated buttons. Page-owned tools can give an agent reliable structured context while keeping data access and actions smaller than the full UI.

I also learned that schemas alone are not enough: runtime validation, optimistic concurrency, idempotent retries, deterministic verification, and visible attribution are all necessary for trustworthy agent workflows. Human-in-the-loop safety is strongest when sensitive authority is absent from the tool surface, not merely discouraged in a prompt.

What's next

Next I would add adapters for other sanitized, page-owned evidence formats while continuing to prohibit raw traffic fetching and arbitrary code execution. I would also add portable evidence bundles, authenticated multi-reviewer workflows with serialized writes, and more deterministic verification strategies for retry and workflow risks. Human approval and final release confirmation would remain explicit boundaries.

Built With

Share this project:

Updates

Submission history