a11y-pr-gate

I am a screen-reader user and an accessibility contributor. This project started from two pull requests I submitted to Hermes WebUI: one improved keyboard access and focus return after Escape in a conversation-actions menu; the other hid inactive dialogs from screen readers. Those changes showed me that static checks alone cannot prove that an interaction still works after a UI change.

I wanted a small tool that checks the behavior before and after a pull request: Does focus return to the right place? Is a dismissed dialog really gone from the accessibility tree?

What I learned

I learned that AI can help define what to test, but it should not decide whether accessibility has passed. A model can miss evidence or sound confident without proving anything. For this reason, A11y PR Gate keeps planning and decision-making separate.

How I built it

English is not my first language. I first used a Korean Codex session to discuss the real user problem and compare possible designs. I then wrote a short final design in English and passed it to the main development session.

With Codex and GPT-5.6, I built a small Codex plugin, a JSON schema for interaction plans, and a Node and Playwright runner. Codex creates up to three focused interaction contracts. The runner validates the plan, runs the same checks on the base and head revisions, and saves screenshots, JSON results, and Playwright traces.

The demo intentionally contains two new regressions. The runner finds both and returns BLOCK. The test suite has 16 passing tests.

Challenges I faced

The main challenge was scope. I did not want to claim that an LLM can certify accessibility or replace real screen-reader testing. I kept the project focused on repeatable regressions such as keyboard focus and inactive dialogs exposed to accessibility APIs. Missing or unreliable evidence becomes INCONCLUSIVE, not a pass.

A second challenge was making the project understandable in a short English demo. I used a simple, evidence-first flow: Codex creates a plan, the deterministic runner checks it, and the runner—not the model—makes the final gate decision.

Built With

• codex
• javascript
• json
• playwright