earshot

Earshot scans inaccessible sites, verifies fixes by screen-reader narration, runs safely with Daytona, logs metrics in Braintrust, creates ElevenLabs audio, and emits CodeRabbit-reviewed PRs.

Comment

Earshot

200-character summary

Earshot turns inaccessible websites into reviewable accessibility PRs, verifies fixes by listening like a screen reader, and uses Daytona, Braintrust, ElevenLabs, and CodeRabbit for evidence.

Inspiration

AI is generating websites faster than humans can review them, and a lot of that generated UI is inaccessible. Existing accessibility scanners catch only part of the problem: they can count obvious violations, but they cannot tell whether a blind user can actually understand a page from a screen reader.

Earshot was inspired by that gap. We wanted an agent that does not claim to “make a site compliant,” but instead proposes reviewable accessibility fixes with evidence.

What it does

Earshot scans a real website, runs axe-core, serializes the page into the linear narration a screen reader user would hear, and asks an LLM whether the page is understandable from that narration alone.

Then it proposes accessibility fixes, applies them in a browser context, verifies that the fixes did not introduce new violations, and emits a PR-style patch with before/after evidence.

The core workflow is:

  1. Audit the page with axe-core.
  2. Narrate the accessibility tree like a screen reader.
  3. Judge whether the narration is understandable.
  4. Propose fixes.
  5. Apply and verify the fixes.
  6. Roll back anything that causes regressions.
  7. Emit a reviewable PR with evidence.

The key framing is: an agent proposes; a human disposes.

How we built it

We built Earshot with TypeScript, Node.js, Playwright, axe-core, Zod, and Fireworks.

We used the sponsor stack as part of the actual product flow:

  • Daytona for isolated execution evidence around generated repair work.
  • Braintrust to log before/after remediation metrics.
  • ElevenLabs to turn screen-reader narration into audio for the demo.
  • CodeRabbit to review the emitted patch PR.
  • Playwright + axe-core for scanning and verification.
  • Fireworks for narration judgment and LLM patch planning.

Challenges we faced

The hardest part was not generating fixes. It was refusing unsafe fixes.

Some patches looked plausible but introduced new accessibility failures. Earshot had to rescan the page, compare full violation sets, and roll back patches that made the page worse. That became the main safety principle of the project: the verifier is allowed to reject the agent’s own fix.

Another challenge was emitting a “real” review artifact. We do not own the source repositories for public websites, so Earshot emits a reviewable patch proposal inside our repo with the same kind of evidence a site owner would need: before/after violations, narration changes, flagged items, and patch reasoning.

What we learned

We learned that accessibility repair needs evidence, not confidence.

A useful agent should not say “fixed” just because it changed the DOM. It should say:

  • what improved,
  • what did not improve,
  • what was reverted,
  • what needs human review,
  • and what evidence supports the recommendation.

We also learned that screen-reader narration is a powerful evaluation layer. A page can have fewer axe violations and still be confusing. Conversely, a patch is only meaningful if it improves what a blind user can actually understand.

What is next

Next, we would run Earshot across more targets, improve the patch planner so fewer proposed fixes are rolled back, and connect emitted PRs to real owned demo sites where the patches can land end-to-end.

We would also expand the Braintrust evaluation set so Earshot can compare baseline vs remediated runs across multiple sites, not just one live target.

Built With

Share this project:

Updates