Inspiration

Accessibility tools check rules. A checkout can pass every rule and still lose the sale. In our own test shop, axe-core flagged 2 of 13 checkout defects we built. The other 11 included a promo popup that traps the keyboard and a Pay button made of a <div>. Each of those stops a purchase completely, and the scanner said nothing.

The same pages block AI shopping agents. Agents read a page through the accessibility tree, just as a screen reader does. A checkout that a blind customer cannot finish is one that an agent cannot finish either. In the EU, the European Accessibility Act has applied to online shops since June 28, 2025. So we wanted a test that asks one question: can this customer actually pay?

What it does

AccessiPay sends three simulated customers to buy:

  • Screen reader customer: perceives only the accessibility tree and moves with the keyboard.
  • Keyboard-only customer: sees the page text but has no mouse.
  • AI shopping agent: reads the accessibility tree and acts on controls by role and name, as many browser agents do.

A test passes only when PayPal confirms the order. In our demo shop the customer pays through the PayPal JavaScript SDK (PayPal Buttons and Card Fields). Our server then reads the order back from PayPal Orders v2. Only COMPLETED counts. The customer saying "done" does not.

When a customer gets stuck, an AI fixer reads the exact HTML of that page and the customer's own step-by-step transcript. It returns small find-and-replace patches, and the same customer tries again with the same task.

Recorded results in the demo shop (PayPal sandbox):

  • AI agent, Pay button made of a <div> (axe-core: 0 violations): gave up after 21 steps. After the AI fix (5 changes): paid in 19 steps, PayPal COMPLETED.
  • Keyboard-only customer, promo popup that traps focus (axe-core: 0): gave up after 14 steps. After the AI fix (7 changes): paid in 49 steps, PayPal COMPLETED.
  • Keyboard-only customer, 24 menu links and no skip link: paid, slowly (median 72 steps). After the AI fix (1 change): 52 steps, 28% fewer steps and 15% fewer keypresses.
  • Screen reader customer, clean shop: paid in 36 steps, filling PayPal Card Fields inside their iframes.
  • AI agent given a need, not a product ("keep me dry on hikes, under $100, fast"): chose the Rain Jacket with express delivery, $94.00, PayPal COMPLETED.

The blocked cases were repeated. Before the fix, the keyboard customer gave up after 14, 17 and 19 steps, and the agent after 19, 20 and 21. Every fix was verified by a new run that PayPal confirmed.

Real shops. We sent the same three customers to 55 real online shops. Nobody pays there: each customer stops at the payment step, types no card data, never signs in and uses made-up details. 14 shops turned our automated browser away, and AccessiPay never tries to get past bot protection. On the other 41 shops:

  • Screen reader customer: reached the payment step on 37 of 38 shops
  • AI shopping agent: 30 of 34 shops
  • Keyboard-only customer: 19 of 41 shops

All three customers reached payment on only 15 of the 41 shops. These were the problems met most often:

  • controls with no name or the wrong name (40 shops)
  • keyboard focus lost or sent far away (38)
  • pop-ups that take over the page (36)
  • no feedback after an action such as Add to cart (34)

Shops are published without their names, addresses or screenshots.

Are the findings real? The customers' findings are written by AI, so code with no AI checks each one again against the record of its run. That record holds the accessibility tree at that step, where focus actually went and axe-core on the same page. Of 686 findings on real shops, 365 can be checked this way, and 301 of them (82%) are confirmed. The other 321 are marked as needing a person to judge from the replay.

What the owner gets. Every test produces:

  • a list of fixes ranked by who they block, each with the run that proves it
  • ready code (fix.patch)
  • a screenshot with the problem circled
  • a tagged, accessible PDF report

On the owner's own shop, all the way to PayPal. An owner proves the site is theirs with a file or a DNS record and, if the shop sits behind bot protection, adds one allow rule for AccessiPay's key header. They can then connect a staging copy of the shop and its PayPal sandbox app. The customers pay there with a PayPal test card, and a run passes only when the owner's own sandbox reports an order created during the run as COMPLETED. Payments may go only to a PayPal checkout loaded with that sandbox app's Client ID; everything else is blocked, and live keys are refused. On a separate staging shop we built to test this flow, all three customers paid and the shop's sandbox confirmed all three orders.

A separate check reads what a shop tells AI assistants, such as robots.txt rules for agents like ChatGPT-User or Claude-User, llms.txt and product data.

Honest limits.

  • The customers are simulated, not people with disabilities. They do not replace testing with disabled users.
  • This is not a WCAG audit or a certificate.
  • We wrote the 13 demo-shop defects ourselves, which is why we also tested real shops.
  • On real shops, "reached payment" means the payment form was on screen, not that the payment would have succeeded.

How we built it

  • PayPal. The demo shop pays through the PayPal JavaScript SDK with components=buttons,card-fields. Our server creates and captures the order with Orders v2 and then reads it back. That read-back is the oracle: a run is "paid" only when PayPal returns COMPLETED. Calls go through a token bucket, because the sandbox blocks an IP after about 50 requests a minute.
  • Customers. Playwright drives a real Chromium. Each persona gets its own view of the page (accessibility tree, or visible text plus focus) and its own allowed actions. The engine records every step: what the customer perceived, what it decided, what happened, a screenshot and the checkout stage.
  • Safety on real sites. The engine will not:
    • pay, place an order or type card data
    • sign in or enter passwords
    • send anything to a payment provider (a network guard blocks it)

It also reads robots.txt, stops at bot checks and HTTP 403/429, and waits at least 3 seconds between actions.

  • Fixer. It gets the HTML exactly as the browser received it plus the transcript, and returns patches that must each match the page exactly once.
  • Models. The engine is model-agnostic: a "brain" decides each step, and the fixer is a separate call. The recorded runs and fixes on the site come from Claude, driven by Claude Code agents. Live runs in the public demo use NVIDIA Nemotron 3.5 Super through the NVIDIA API, with the same prompts.
  • Scanner side. Every run ends with an axe-core scan of the same pages, so the console shows "what the scanner says" next to "what happened".
  • Console. React and Vite. Live and replayed runs with a step-by-step "theatre", analytics, per-shop analysis pages and owner reports. AG Grid Enterprise powers the runs, defects and shops tables: row grouping, set filters and master/detail.

Challenges we ran into

  • PayPal card fields live in iframes. At first our keyboard and screen reader customers could not get inside them. Real users can, so the engine had to as well.
  • Real shops fight automation. Bot walls often look like accessibility failures, for example an Add to cart button that never works. Every run that did not reach payment was reviewed by hand before it was published. A run was marked "not counted" when the fault was ours, and "turned away" when the shop blocked the browser.
  • Telling the truth about AI findings. That is why every finding is checked again without AI and shows whether the record confirms it.

Accomplishments that we're proud of

  • A pass mark no model can fake: a COMPLETED PayPal order.
  • Two checkouts that were impossible and became possible after an AI fix, while the scanner reported zero violations on both.
  • One number that holds up across 41 real shops: the keyboard-only customer reached payment on fewer than half of them.

What we learned

Accessibility and agentic commerce are the same problem seen from two sides. The fixes that let a screen reader user pay also let an AI agent pay.

What's next for AccessiPay

  • Customer types for voice control, 400% zoom and an invisible focus indicator.
  • Re-running a merchant's connected staging shop after every release, so a checkout change that stops a customer is caught before it reaches real buyers.
  • A pre-launch check in PayPal's integration flow: before a merchant switches from the sandbox to live payments, AccessiPay runs the three customers on the staging checkout and shows who could not pay, with the fix.

Built With

  • accessibility
  • ag-grid
  • ag-grid-enterprise
  • agentic-commerce
  • anthropic
  • axe-core
  • chromium
  • claude
  • claude-code
  • express.js
  • javascript
  • nemotron
  • node.js
  • nvidia
  • paypal
  • paypal-card-fields
  • paypal-javascript-sdk
  • paypal-orders-v2
  • paypal-sandbox
  • playwright
  • react
  • render
  • vite
  • wcag
Share this project:

Updates

Submission history