odyssey

Inspiration

Automated accessibility scanners are valuable, but they usually evaluate isolated rules—not whether someone can complete an entire task.

A page can pass conventional checks while an essential action remains unreachable. For example, a styled <div> may look exactly like an Add to cart button to a sighted developer while exposing no semantic activation role or keyboard-reachable action.

That gap inspired Blind Spot:

Automated scanners test rules. Blind Spot tests whether a task remains reachable through an accessibility-derived interaction surface.

Blind Spot is not a replacement for disabled-user testing, assistive-technology testing, axe, Lighthouse, or Playwright. It provides an earlier, task-level signal that helps developers find critical barriers before shipping.

What it does

Blind Spot gives an agent a constrained set of semantic actions derived from the application’s currently rendered state.

Our demonstration begins with the task:

Add the cheapest keyboard to the cart.

The agent can read the page and navigate its semantic model, but the cheapest product uses a pointer-only <div> as its Add to Cart control. Because Blind Spot cannot find a defensible semantic activation action, it does not expose activate_current.

The audit then produces an evidence-backed finding containing:

The blocked task step
The affected element
Its observed role and accessible name
Keyboard-focusability evidence
The detector and confidence level
The reason the action was withheld
A suggested remediation

The developer applies the demonstrated semantic repair, replacing the inaccessible control with a native button. Blind Spot immediately recompiles the capability surface, and activate_current appears in the WebMCP registry.

The same task is then repeated. This time, the agent activates the repaired control, adds the product to the cart, and completes the audit successfully.

The visible transition from:

Task blocked
activate_current unavailable
Cart: 0

to:

+ activate_current
Task passed
Cart: 1

is the core of Blind Spot.

How we built it

Blind Spot is a React and TypeScript web application with four connected layers:

Rendered application
        ↓
Semantic adapter
        ↓
Capability compiler
        ↓
Dynamic WebMCP registry
        ↓
Agent task attempt and evidence report

The semantic adapter examines browser-observable information from the live page, including native semantics, accessible names, focusability, and application-declared intended actions.

The capability compiler converts that evidence into the actions the agent may use. It also creates diagnostic action-gap records when an intended human action has no corresponding semantic exposure.

The generated tools are registered through the browser-native WebMCP API:

document.modelContext.registerTool(...)

Our core tool surface includes:

start_accessibility_task
read_current_context
move_cursor
inspect_current_evidence
report_task_blocker
finish_accessibility_task
activate_current, registered only when the current action is defensible

Each registration has its own AbortController. When the page state changes and a capability is no longer valid, Blind Spot aborts that registration. When a repair makes an action available, the registry adds the corresponding tool.

Tool results include structured state and evidence so the agent can verify what happened instead of relying on a generic success message.

We also built a visible mission-control interface showing:

The agent’s constrained perception
The current semantic cursor
Available WebMCP tools
Tool invocation history
Registry revisions and diffs
Evidence-backed findings
Before-and-after audit status

A scripted audit mode exercises the same registered tools through getTools() and executeTool(), allowing the workflow to be inspected without replacing WebMCP with a mock.

Why WebMCP

WebMCP is especially well suited to this problem because accessibility depends on live application state.

Source HTML alone cannot reliably represent computed visibility, focus state, portal content, dynamic relationships, or the exact mid-workflow page the developer is inspecting. Blind Spot operates inside that same rendered tab and updates its tool surface as the application changes.

The WebMCP registry is the executable test surface—not the source of truth by itself. Blind Spot’s capability compiler decides which actions are supported by current evidence, the registry enforces that decision for the constrained workflow, and the captured evidence explains every finding.

This also creates a collaborative human-agent loop:

The human provides a real task.
The agent attempts it through the constrained tool surface.
Blind Spot identifies where the path becomes unreachable.
The human repairs the application.
The WebMCP surface changes immediately.
The agent repeats and completes the same task.

Challenges we faced

The greatest challenge was keeping our claims technically honest.

A webpage cannot guarantee that an agent has no other browser capabilities, and it cannot directly reproduce everything NVDA or VoiceOver would announce. Blind Spot therefore does not claim to simulate the lived experience of a blind user or certify accessibility compliance.

Instead, it runs a documented, constrained audit protocol and records which semantic actions were available at each step.

We also learned that:

Missing tools are not sufficient evidence by themselves.
DOM event listeners cannot always be discovered reliably, especially with delegated React events.
Accessible semantics are not identical to the browser’s complete platform accessibility tree.
Synthetic keyboard events cannot prove every aspect of native keyboard behavior.
Dynamic tool registration requires careful lifecycle and concurrency handling.
The auditing product must itself be accessible.

These constraints led us to pair semantic evidence with an application-declared action inventory. This lets Blind Spot compare what the application intends people to do with what is actually exposed through its semantic interaction surface.

What we learned

WebMCP is more than a convenient way to let an agent click interface controls. Its dynamic registration model can express temporary, state-dependent capabilities.

We learned that tool absence can be meaningful when it is produced by a transparent compiler and accompanied by reproducible evidence. We also learned that narrowly scoped, high-confidence findings are more useful than broad accessibility claims that the browser cannot prove.

Most importantly, we found that human-agent collaboration can improve developer tooling—not only end-user automation. The agent attempts the task, the developer makes the judgment and repair, and both participate in validating the result.