-
-
The 4 checkout bugs most stores never test
-
How it works: PayQA forces PayPal failures through the adapter, compares with PayPal's record, Claude explains
-
Chaos Mode on the buggy store: 5 of 7 scenarios fail, 14/100
-
Claude explains each bug's impact and shows the code fix
-
Browser Mode catches the coupon bug: $60.07 on screen, $66.75 at PayPal
-
Fixed store: every scenario passes, 100/100, cleared for go-live
-
Works with any Node + Express store: one file, one line
Inspiration
I'm a QA engineer. Most stores only test the happy path: the card works and the order goes through. The bugs that cost real money happen when PayPal says no: a declined card shown as "Payment successful", an order marked paid before PayPal confirms, a retry that charges twice, a coupon that never reaches PayPal. These are hard to test because you need PayPal to fail on demand. PayPal's sandbox can do that, but almost nobody uses it
What it does
PayQA tests a store's PayPal checkout and tells you if it's safe to go live.
- Chaos Mode runs 7 scenarios against the real PayPal sandbox. It uses negative testing to make PayPal decline cards, refuse transactions and fail mid-payment, then reads the order straight from PayPal and compares it with what the store did.
- Browser Mode uses Playwright to shop like a real customer: it opens Chrome, logs into the PayPal sandbox popup and pays. It checks that the total on screen matches what PayPal charges, tests cancelling, and saves screenshots as evidence.
- Claude analysis: for every failure, Claude explains the business impact and shows the code fix, using the evidence from that run.
- You get a readiness score out of 100 and a "Cleared for go-live" or "Not ready" stamp.
The demo store (Lanka Crafts) has 4 planted bugs and scores 14/100. The fixed version scores 100/100.
How we built it
- Node.js + Express for the PayQA server, dashboard and demo stores.
- PayPal Orders v2 API, OAuth, the PayPal JS SDK, and sandbox negative testing through the
PayPal-Mock-Responseheader (INSTRUMENT_DECLINED, TRANSACTION_REFUSED, INTERNAL_SERVER_ERROR). - PayQA Adapter: one file a merchant drops into their Express store, plus one line added to their PayPal calls. It's off by default, sandbox only and token-protected, so PayQA can trigger failures without touching real payments.
- Playwright for the browser checkout tests.
- The Claude API (Sonnet) for the analysis, with built-in explanations as a fallback.
- Hosted on Render.
Challenges we ran into
- Making PayPal fail on demand inside someone else's checkout without changing their code much. That's where the adapter idea came from.
- The PayPal popup in Playwright: hidden duplicate buttons and changing login layouts. I fixed it with visible-only selectors, step labels, retries and screenshots on error.
- Free hosting can't run Chrome, so on the live demo Browser Mode runs locally only. Chaos Mode works fully online.
Accomplishments that we're proud of
It catches all 4 planted bugs on the real PayPal sandbox, and Claude explains each one with a working fix. The same tool connects to a second, minimal store with only the adapter added.
What we learned
PayPal's sandbox negative testing is very powerful and underused. Also, comparing against PayPal's own record of the order catches far more than checking what the store's UI says.
What's next for PayQA
- Claude pointing to the exact lines in the merchant's own code.
- AI-driven Browser Mode that explores a checkout by itself.
- Adapters for more frameworks beyond Express.
Note for judges: the live demo runs on free hosting, so the first load may take about a minute to wake up.
Built With
- anthropic
- claude
- express.js
- javascript
- node.js
- paypal
- paypal-js-sdk
- paypal-orders-api
- playwright
- render