Sightline

The human-approval layer between an agent and a live page. The agent finds what holds a page back, drafts the fix, and stops until you say yes — inside the page, through WebMCP.

Comment

Inspiration

Most "AI fixes your site" tools edit a copy of your page somewhere you cannot watch, then report "done". We wanted the agent on the live page, in front of you, through declared tools, with a hard stop before any change.

WebMCP gives a page that control. The page declares the tools an agent may call, and apply_fix can keep a tool call open until you approve.

What it does

Sightline is an approval layer between a browser agent and a live page. The agent finds what holds the page back (accessibility, SEO, performance), drafts the fix, and waits for your click before the DOM changes.

Nine imperative WebMCP tools cover three rule packs: accessibility (axe-core), SEO and performance. One engine, one gate.

  • list_packs, then scan_page: 23 problems on the bundled fixture (14 accessibility, 5 SEO, 4 performance), overall score 54.
  • highlight_issue: you and the agent look at the same live element.
  • propose_fix refuses to invent human-facing content. For alt text, labels, roles, titles, descriptions and link text it returns needs_input plus page context, and the agent writes the text. The engine measures contrast, heading level and image dimensions from the live DOM. Each patch carries evidence[] and authoredBy.
  • apply_fix stages the patch in a visible approval sheet and returns an unresolved promise until you click Approve or Skip. After approval it re-scans and returns the new score.
  • revert_fix reverses approved changes. export_patch hands back approved work as a diff or a report.

On the demo: 54 to 100 overall, 100 / 100 / 100 per pack, 23 approved changes, no planted answers.

Why WebMCP fits

Fixing a page is a loop: inspect the element, weigh the impact, propose a bounded change, apply it with judgment, verify. The loop is the same for a missing alt text, a generic title or an image without dimensions. A browser agent working from screenshots has to guess at that loop through unstable DOM controls. Sightline exposes each step as a WebMCP tool with typed inputs and safety annotations.

The page registers the nine tools as state allows. Read tools carry readOnlyHint. Tools that return audited page content carry untrustedContentHint. Mutations wait until you confirm the before/after DOM in the approval sheet. You and the agent share one page, one focused element, one issue list and one score.

How it improves the experience

You stop hopping between scanner, chat, editor, browser and report. The agent prioritises and explains barriers by stable issue id. You see the highlighted element and keep the final say on each mutation. Sightline re-scans after each approved fix, lets you undo any one of them, and exports the approved set as a diff or report.

The judging demo is deterministic: 23 real problems, per-pack and overall scores, and a visible path from 54 to 100. The agent authors nine of the fixes; the engine measures fourteen. The fixture contains problems and nothing else.

What you and an agent can do together now

  • Run one approval workflow across accessibility, SEO and performance instead of three tools with three reports.
  • Point at the same live element without translating between report, chat, selector and source file.
  • Turn audit output into reviewable DOM patches without running model-generated code.
  • Pause an agent's tool call inside the page until you decide.
  • Check each claim against the changed DOM instead of trusting "fixed".
  • Export approved changes with rationale and a scope note.

How we built it

Sightline uses the imperative document.modelContext.registerTool() API. Each tool ships a JSON Schema, a plain-language description and annotations. Registration in Chrome uses AbortController cleanup and state-driven discovery, so tool-change events fire without re-registering active tools.

One TypeScript engine drives the manual controls and the agent callbacks, so the two paths cannot drift. Scans run in an isolated 900 px audit frame, so results match on any viewport. propose_fix splits the work: the agent authors alt text, labels and ARIA roles (validated as plain text, length-bounded, roles allow-listed); the engine computes the passing colour from measured WCAG luminance and derives heading levels from the live outline. Patches carry evidence (measured ratios, current attributes) instead of a confidence number. apply_fix returns an unresolved promise while the approval sheet is open, then resolves with the new score. The bundled "Watch the agent work" demo is a WebMCP client that calls getTools() and executeTool(), the same path an external agent takes.

Stack: React, TypeScript, Vite, axe-core, imperative WebMCP, Cloudflare Pages. No backend, no auth, no model API, no secrets.

Challenges

  • Holding a tool call open across a human decision. apply_fix resolves after approval and re-scan. Chrome 151's cancelInvocation does not fire the callback's AbortSignal, so the engine adds a 120 s timeout and a Skip path.
  • Not cheating. Our first fixture carried planted answers in data-fix-* attributes. We deleted them and made the engine return needs_input, so the agent reads the page and writes. The fake confidence score became measured evidence.
  • Viewport-independent audits. The 900 px audit frame gives an agent on a phone-sized viewport the same 23 issues a desktop judge sees.
  • axe-core across realms. axe.run(HTMLElement) fails across the iframe boundary. We pass a selector string, and import axe.min.js?raw because production tree-shakes axe.source.

Accomplishments

  • We verified the full demo (23 to 0 across three packs) with an external agent through Chrome's native WebMCP surface (CDP WebMCP.*). The launch video shows that run.
  • The "Watch the agent work" button is a WebMCP client (getTools(), executeTool()). No side channel.
  • A new rule pack is one file with { scan, fixers } plus its id in the PackId union and the engine's pack list. The tool surface, the gate, undo and export come with it.
  • npm run smoke replays the verification: 15 invariants through Chrome's WebMCP DevTools domain. CI runs it against each build in headless Chrome; it also runs against the live URL.
  • A Spec Kit constitution with six principles. The first: Approval Before Mutation. A source-level test checks that the DOM-mutating function has two call sites, approval and replay of approved history.
  • Vitest covers the content validator, the WCAG contrast maths and the approval state machine, including a regression test for two agents overlapping the post-approval re-scan (a race we found in review). CI adds a bundle budget and a no-planted-answers check.

What we learned

A page can declare where the human sits in a workflow and make the agent wait there. readOnlyHint, untrustedContentHint, staged registration and AbortSignal turn that from a convention into a contract.

What's next

  • Import any page (CSP and an import UI).
  • Gate revert_fix behind the same approval. Add Playwright coverage of the native tool flow.
  • More packs on the same engine: security headers, structured data, i18n hygiene.

Scope

Sightline demonstrates remediation with a human in the loop. It does not claim automatic WCAG compliance. Automated checks support manual and expert accessibility testing; they do not replace it.

Built With

Share this project:

Updates