Lastest: QA reports with WebMCP site tools

Lastest verifies web releases with real-browser runs and visual diffs. Every page now exposes typed WebMCP tools: agents read public QA reports without login, and run, review and approve in-app.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Lastest: QA reports with WebMCP site tools

Lastest: a QA platform that hands its own controls to the browser's agent

Inspiration

Lastest is a visual-regression / release-verification platform: it drives a real browser through your web app, captures every step, diffs screenshots, checks accessibility, and produces a verdict. Agents already talk to it over a remote MCP server (29 tools, OAuth 2.1). But the place where a human actually looks at a result is a web page: a run report, a build, a public /r/<slug> share. When ChatGPT's browser and Codex started consuming WebMCP "site tools" (2026-08-25), the missing piece became obvious: the page you are looking at should expose the same typed tools the remote server does, scoped to what is on screen, so the agent beside you never has to scrape a screenshot or guess a button.

What it does

Every Lastest page registers typed tools on document.modelContext (with a polyfill fallback for browsers that have not shipped it yet):

  • Public report pages (/r/<slug>, no login): lastest_report_summary, lastest_list_visual_changes, lastest_list_failing_steps. An agent visiting a shared QA report can read the verdict, the failing steps and the pixel diffs as structured data instead of pixels.
  • Signed-in app (dashboard, Verify, tests): a curated 16-tool surface out of the 29 on the MCP server: list projects/builds, QA summary, coverage, failing tests, change map, run tests, review a build, approve/reject diffs, publish a share, heal a test.
  • Route-scoped registration. The set narrows and re-binds as you navigate: the build page contributes its buildId, the test screen its testId, so the agent gets ids from the page, never from its own guesswork. Tools unregister on unmount.
  • Consent on every mutation. Write tools (run_tests, approve_diffs, heal_test, ...) call requestUserInteraction() where available and always go through Lastest's own consent dialog that names the exact action. A dismissed dialog is never consent.
  • Same auth as the human. Tools call a cookie-authed, same-origin /api/mcp/session bridge (CSRF-guarded with a custom header), so the agent can only ever act with the signed-in user's permissions. Public share tools are slug-scoped and never consult a session.

How we built it

  • src/lib/webmcp/: feature detection (document.modelContext, falling back to the deprecated navigator.modelContext and to @mcp-b/webmcp-polyfill), registry/allowlist with readOnlyHint annotations, argument binding from route context, consent wrapper, result shaping.
  • src/app/api/mcp/session/route.ts: JSON-RPC bridge that reuses the exact MCP server the remote endpoint uses (createServer() over InMemoryTransport), so the WebMCP surface can never drift from the MCP surface. A test fails if an allow-listed tool disappears from tools/list.
  • src/app/api/webmcp/share/[slug]/route.ts: login-free read-only tools for public reports.
  • React provider + route-context components mounted in the app layout and the share page.
  • 30+ unit assertions plus a Playwright e2e suite that injects a recorder modelContext and checks registration, route narrowing, disposal and the consent gate.

Prior work vs. new work

Lastest existed before the challenge (visual regression, MCP server, share pages). All WebMCP work was created inside the submission window: design doc 2026-08-27, feat(webmcp) PR #119 and follow-up #123 merged 2026-08-31, e2e suite and polyfill/consent slices after that. The git history in las-team/lastest is the audit trail; docs/design/webmcp.md records the design and its dated status.

Challenges

  • Spec churn: navigator to document, unregisterTool to AbortSignal, provideContext removed. Everything API-facing is confined to one module with a feature-detect shim that handles both shapes.
  • Prompt-injection safety: destructive/admin tools are on a never-register list enforced by a test; every mutation is gated by consent.
  • Keeping one source of truth: the browser tools are descriptor-driven from the real MCP server rather than a second hand-written tool list.

Try it

What's next

Consume WebMCP tools on the pages Lastest tests (agents verifying an app through its own declared tools), and surface tool-call traces in the run report.

Built With

Share this project:

Updates

Submission history