Inspiration
PayPal keeps shipping a faster checkout (JavaScript SDK v6, PayPal on the cart page, better mobile placement), but every improvement only helps when a merchant changes their own code. Most small stores never do: they don't know their integration is outdated, upgrading needs a developer, and testing it needs real payments. Shoppers get a clunkier checkout and stores lose sales. We wanted an agent that does the whole upgrade for them, and proves it is safe.
What it does
Give CheckoutPilot a store URL. It works in four steps, then re-measures:
- FIND – walks the store like a shopper (product → cart → checkout, desktop and mobile) in a real browser. Rule checks and Gemini vision read every page and screenshot.
- RANK – orders the problems by business impact and gives a health score, with every assumption tied to a cited PayPal source. Gemini also writes a plain-language summary for the store owner.
- FIX – a code agent writes the upgrade (PayPal SDK v5 → v6, PayPal on the cart page, mobile placement) on a copy of the store, shows a diff for every file, and opens the copy in a browser to self-test it.
- PROVE – an automated buyer makes two real PayPal Sandbox payments, one on the original store and one on the fixed copy, and checks both orders with the PayPal **Orders API.
- RE-MEASURE – the same scanner runs on the fixed copy: on our demo store the health score goes from 21 to 76 (+55), 3 of 4 issues fixed, now on SDK v6.
The merchant sees the issues, the diff and the payment proof, and only then approves. Nothing changes silently. The dashboard also has a sortable findings table (AG Grid) with CSV export, a before/after comparison and a "what could this be worth" calculator using the merchant's own numbers.
How we built it
- Scanner: Python + Playwright (Chromium), desktop and mobile viewports, honest user agent, public pages only.
- AI engine: deterministic rule checks + Gemini vision on screenshots; Gemini also writes the owner summary, with guardrails (no invented numbers or money claims, template fallback).
- Impact ranker: severity × category weights, health score, cited assumptions.
- Code agent: patches a copy of the store, follows a verified v6 recipe, and runs a browser self-test before anyone sees the fix.
- PayPal: Sandbox REST (OAuth, Orders v2 create / capture / get), JS SDK v5 in the "before" store, SDK v6 (createInstance, findEligibleMethods, one-time payment session) in the "after" store.
- Backend: FastAPI. Dashboard: React (Vite + TanStack) with AG Grid.
- Hosting: dashboard on Vercel, backend as a Docker web service on Render.
- Tests: 330+ pytest and 40 Vitest tests.
Challenges we ran into
- SDK v6 is new: our first upgrade loaded the wrong components and showed a $0.00 total, so we built a browser self-test that catches broken fixes before the merchant sees them.
- Sandbox payments only worked with a United States Personal buyer account; Business accounts or India-to-India payments fail.
- Real stores render slowly, which gave blank screenshots and false findings, so the scanner now waits for each page and only judges pages it actually saw.
- Fitting Chromium, two demo stores and the payment verifier into a free 512 MB server.
Accomplishments that we're proud of
- A full loop that actually runs: find → rank → fix → prove with real Sandbox orders → re-measure.
- The agent never claims success it hasn't shown: every fix is tested in a browser and with real payments, and the AI's text is checked before it is displayed.
What we learned
How PayPal's v5 and v6 SDKs differ, how to use the Orders API to verify payments, how to keep an AI agent honest with guardrails and fallbacks, and how to work as a five-person team with feature branches and pull requests.
What's next for CheckoutPilot
Opening the fix as a real GitHub pull request on the merchant's repository, support for more e-commerce platforms, and tracking real conversion change after an upgrade.
