Inspiration
95.9% of the top million homepages fail accessibility checks. The tools everyone relies on catch about half of that, and almost none of the failures that actually block a task — a size picker that a screen reader announces as plain text, nameless buttons heard as silence. That's not an edge case, it's the checkout. We wanted the test to be what a blind user actually experiences: can the task be finished by ear? Not "does the markup pass a rule."
What it does You give Blind-Run a URL and a task in plain English
"buy the cheapest item". It drives a virtual screen reader through the page and records every phrase announced. If the task dead-ends, an AI auditor names the exact WCAG barrier quoting the transcript, a fixer writes 3 candidate fixes, and all three run in parallel inside Nebius Sandboxes branches — applied, served, and re-listened to there. Only a fix whose task completes by ear ships, as a PR with the before/after transcript. The full loop: 7,414 tokens, about one cent.
How we built it Every model call runs on Nebius Token Factory:
Nemotron-3 Super plans the walk and audits the transcript, Nemotron-3 Ultra writes the fixes. Tavily retrieval grounds the fixer with the current W3C pattern (the PR cites the URL it fetched) and links every finding to the official WCAG doc. The whole thing sits behind 7 swappable ports with fake adapters, so 19 offline tests cover the pipeline with no keys. The pass/fail oracle is a string check on the transcript — never a model opinion. There's a dashboard that auto-plays the recorded run with a stage timeline, and a benchmark over 21 planted bugs with Wilson CIs.
Challenges we ran into Recording the flagship demo took 10 takes.
The worst one: every fix branch kept failing and the error made no sense — until we added md5 fingerprints inside the sandbox and discovered all branches were patching the WRONG page (index.html instead of product.html) because one adapter wasn't receiving the page context. One line fixed it. Also: models cannot write unified diffs or JSON-escaped diffs — we settled on verbatim SEARCH/===/>>> blocks with a worked example from the file's own lines, and built tolerant fallbacks for when they re-indent, merge all candidates into one unclosed block, or reply with reasoning prose instead. And we found two real Token Factory bugs along the way — a pattern keyword anywhere in a json_schema returns HTTP 500 (bisected keyword by keyword), and reasoning can eat the entire output budget and return empty content. Both documented with repro steps in FEEDBACK.md.
Accomplishments that we're proud of The recorded demo is a real run:
the winning fix is the W3C fieldset/legend/native-radio pattern, chosen not by a human but by verification — the screen reader heard "radio, S, not checked… Added to cart" inside the Nebius branch. Two of three candidates failed honestly; we kept that in the demo on purpose. The benchmark is honest too: axe alone catches 0 of the 10 blockers invisible to rule engines; our ear lane runs 100% precision with zero false positives on clean controls. And it works outside our fixtures — on a real product page we didn't author it found 2 genuine blockers; on Wikipedia's accessibility article it reported 0 findings twice.
What we learned Verification has to be deterministic.
An LLM that writes the fix will also "confirm" the fix — so the oracle has to be something that can't hallucinate: a transcript string check. Also: degrade, don't crash — search falls back to cache, sandboxes to local, a 3,000-character product description gets clipped at the schema boundary instead of failing the run. And an honest escalate ("no candidate passed, here's the evidence") is a feature, not a failure. The system refusing to ship an unverified fix is the whole point.
What's next for Blind Run Near:
real-reader confirmation (VoiceOver/NVDA) behind the same Ear port, multi-page checkout flows, auditor recall tuning on the axe-invisible barriers. Beyond: org-wide GitHub App gating and an MCP server so coding agents hold their own UI to the ear — the task → experience → fix → prove loop generalizes to any oracle you can render as text.
Built With
- axe-core
- css
- fastapi
- github-actions
- guidepup-virtual-screen-reader
- html5
- httpx
- javascript
- nebius-sandboxes
- nebius-token-factory
- node.js
- nvidia-nemotron
- playwright
- pydantic
- pytest
- python
- server-sent-events
- tavily
- tenacity
- typer
- uv
- uvicorn
Log in or sign up for Devpost to join the conversation.