Sherpa

Your checkout is losing sales and you can't see why. Sherpa's AI shops your store like a customer, fixes your PayPal checkout, proves it with a real payment, and opens it to AI shoppers.

Comment

Inspiration

Most people who run an online store aren't payment experts. They're bakers, makers and people who sell gear they love. They add a PayPal button, try it once, see that it works and get back to running their business.

But checkout can break in quiet ways that nobody notices:

  • a PayPal button that doesn't show up on phones
  • a PayPal button so far down the page that shoppers never see it
  • no Venmo or Pay Later for shoppers who expect them
  • an old PayPal script that's no longer recommended
  • prices the browser can change before payment
  • payment notifications the store never checks are real
  • a checkout that shows a blank page when anything goes wrong

Each one means someone who wanted to buy and couldn't. The owner almost never finds out, because a customer who gives up doesn't write in to explain why. They just leave.

We wanted to know if this was real, so we looked. We scanned 186 real online stores. Every single store that loads PayPal on its public pages, 16 out of 16, still runs the old PayPal script, and 14 of them show no Pay Later messaging where shoppers decide to buy.

None of these are hard to fix for someone who knows PayPal well. Most small stores just don't have that person.

At the same time, a new kind of shopper is arriving: AI assistants that shop on someone's behalf. "Find me a birthday cake for 12 people, delivered Saturday, under $90." An assistant can only buy from a store that lists its products in a form it can read and offers a way to check out through PayPal. Almost no store has that yet.

We asked a simple question: what if every store had a PayPal expert who could find the problems, fix the code, and prove the fix works, in minutes?

That's Sherpa. A sherpa guides people up a hard climb. For online stores, the climb is checkout.

What it does

Give Sherpa a store's web address and its GitHub repo. Then watch it work.

1. Audit: it shops your store like a real customer

Two AI shoppers open real browsers, one on a desktop and one on a phone, and try to buy something with PayPal. You watch them live as they scroll, tap, fill in delivery dates and get stuck. Behind them, Sherpa runs exact checks: which PayPal script is loaded, where the button sits, whether Venmo and Pay Later appear, whether the server trusts prices sent from the browser, and whether payment notifications are verified.

It also checks the things a PayPal engineer looks for in a code review:

  • Does a declined card end the sale, or can the shopper pick another way to pay?
  • Can a timed-out payment call be retried safely, without charging twice?
  • Does the store wait for a payment that is still pending, or does it ship the order anyway?
  • Does it ignore a payment notification it has already processed? PayPal retries them up to 25 times.

You get a score out of 100, screenshots, a replay of each shopper's visit, plain-English findings, and an estimate of the PayPal revenue those problems put at risk each month.

2. Fix: it writes the code and opens a pull request

A coding agent opens the store's code, and you watch the files change live. It moves the store to PayPal's current script, puts PayPal and Venmo where shoppers can see them, adds Pay Later messaging, sets prices on the server, verifies payment notifications, recovers from declined cards, makes payment calls safe to retry and handles pending payments.

It isn't allowed to stop when it thinks it's done. The build has to pass and Sherpa's own checks have to agree. Then it opens a real pull request on GitHub, written so a store owner can understand it.

3. Verify: it proves the fix, then tries to break it

Most tools stop at "here's a fix." Sherpa keeps going. It deploys the pull request as a private preview of the store, sends both shoppers back in, and completes real PayPal sandbox payments on desktop and on a phone, confirmed by PayPal's Orders API.

Then it attacks the store, before and after the fix:

  • Price tampering: it asks the store for a $1.00 order for a $349 tent, then reads the order back from PayPal. Before the fix, PayPal created the order for $1.00. After it, the order was $349.00.
  • Forged payment notification: it sends a fake "payment completed" message to the store. Before the fix the store accepted it. After it, the store rejected it.

Security fixes get proven, not just promised. The before-and-after scorecard and the attack results are posted on the pull request.

4. Agent-ready: it opens the store to AI shoppers

Sherpa adds a product catalog that AI assistants can read and the PayPal Cart API endpoints they use to check out, and tests every endpoint. Then an AI buyer shops the store the way a person's assistant would: it reads a request in plain English, answers the store's questions ("Which delivery date?") and pays through PayPal.

Sherpa also turns every agent-ready store into an MCP server, the open standard AI assistants use to connect to tools. Any assistant that speaks MCP can now search the store, build a cart and get a PayPal payment link, with the person always approving the payment on PayPal's own page. We tested it with the official MCP client, and you can try it from the console with one click.

Beyond the four stages

  • Autopilot: one click runs all four stages back to back.
  • Agent Shelf: shows what an AI shopping assistant finds everywhere else for the same products, using live prices from Channel3, and where your price sits.
  • Fleet scan: agencies and platforms paste up to 25 store addresses and get them ranked by checkout health in seconds, in an AG Grid table.

Why Sherpa stands out

  • It does the work. It finds the problem, writes the fix and opens the pull request.
  • It proves the result. Real PayPal payments on desktop and phone, and real attacks that fail after the fix.
  • It's measured, not claimed. A benchmark plants 11 kinds of PayPal flaws across 126 store variants. Sherpa finds exactly the planted flaws in 126 out of 126, with zero false alarms.
  • It works beyond its own demos. We ran it end to end on PayPal's own official sample checkout, code we didn't write. It found the real problems, upgraded it in a pull request, completed a payment and kept the AI catalog to the one item the sample really sells.
  • The numbers are honest. Plain code decides every score; the AI only explains. When a cloud browser fails, Sherpa says "not tested" instead of blaming the store.
  • It's ready for how people will shop next, through PayPal's Cart API and MCP.

How we built it

  • Gemini 3.8 Flash on Vertex AI powers every AI part of Sherpa:
    • computer use for the shoppers, who see the screen and click like a person
    • structured output for findings that code can check
    • function calling for the coding agent and the AI buyer
    • Everything signs in with a Google Cloud service account, with no API keys in the code.
  • PayPal (sandbox): JS SDK v6, Orders API v2, webhook signature verification, Venmo, Pay Later, the Cart API, the Agent Toolkit, and PayPal's AI Toolkit best-practice guides, so the coding agent follows PayPal's own advice. Every PayPal rule Sherpa checks is grounded in PayPal's docs or its published API schema.
  • KERNEL cloud browsers, driven with Playwright, with live view and session replays.
  • Channel3 for live product data in Agent Shelf and for finding real stores to study.
  • AG Grid for the fleet scan table.
  • The MCP standard (Streamable HTTP) for connecting any AI assistant to an agent-ready store.
  • Google Cloud: Cloud Run, Cloud Build, Artifact Registry, Firestore, Cloud Storage and Secret Manager.
  • Next.js and React for the console, FastAPI (Python) for the agent, and 102 automated tests, including the benchmark.
  • Three test stores: Ember & Oak (outdoor gear, plain HTML and Express), Lumen Bakehouse (cakes, Next.js) and PayPal's own official sample checkout.

We were careful with the revenue estimate too. When two problems hit the same shopper, that shopper is only lost once, so Sherpa never counts the same lost sale twice.

Challenges we ran into

  • Testing found our own bugs, which was the point. Our benchmark's first run scored 88 out of 126 and exposed two blind spots in Sherpa's checks. Running Sherpa on PayPal's sample found six more. All of them are fixed, with tests.
  • An AI agent that invents things. On PayPal's sample, which sells a single $100 order, our agent-ready step once published a whole bakery catalog. It had copied the idea from an example. Now every catalog item's id, title and price must already exist in the store's code, and anything else is flagged in red. The next run published exactly the one real item.
  • Being honest when it isn't perfect. On PayPal's sample, the phone checkout still stops at PayPal's "Click to Continue" popup. Sherpa's own Verify caught this and refused to call the fix perfect: 82, not 100. We'd rather show that than hide it.
  • Phones are hard for AI shoppers. Our phone shopper kept running out of steps on a delivery-date picker, because it typed "11202026" into a field that ignores typing. Sherpa now sets date fields directly. Lumen Bakehouse went from failing on phones to a completed phone payment.
  • Cloud browsers sometimes disconnect. At first Sherpa blamed the store for that. Now it keeps every PayPal order it saw, retries once, and marks the device "not tested" rather than "failed".
  • PayPal's sandbox checkout changes from visit to visit. Sometimes it's a guest page, sometimes a popup that closes itself. We built a sign-in path that handles every version, and the AI never sees the sandbox login details.
  • Keeping the AI honest. A model can sound confident and still be wrong. Code measures and scores, the AI explains, and every claim in our results comes from a real run.

Accomplishments that we're proud of

  • Two very different stores taken from broken to 100, on desktop and on a phone, ending in completed PayPal sandbox payments: Ember & Oak (15 → 100) and Lumen Bakehouse (14 → 100).
  • Attacks that work before the fix and fail after it, verified with PayPal's own Orders API.
  • AI buyers that completed real purchases through each store's new PayPal Cart API.
  • 126 out of 126 on our benchmark, with zero false alarms.
  • PayPal's own sample, upgraded by Sherpa from 71 to 82: SDK v6, a verified webhook, a completed desktop payment and an AI buyer purchase of its one real item.
  • A field study of 186 real stores showing the problem is everywhere.
  • Everything is live and needs no sign-up. Judges can start a real audit and watch it happen.

What we learned

  • Proof changes everything. Once every fix had to end in a real payment and a failed attack, the coding agent got better and so did we.
  • Measure yourself before you claim anything. Our own benchmark found bugs we would never have noticed in a demo.
  • Small checkout details cost real money. A date picker that a phone can't fill, or a button that never appears, quietly loses sales.
  • AI shopping is close. Product feeds, cart APIs, MCP and PayPal checkout all exist today. Most stores just haven't connected them yet.

What's next for Sherpa

  • A GitHub App, so any store owner can install Sherpa on their own repo in a minute.
  • More platforms: Shopify themes, WooCommerce and other popular stacks.
  • More PayPal features in the audit: Fastlane, card fields and saved payment methods.
  • PayPal's Agent Ready protocols (ACP and UCP) alongside MCP, once merchants are approved.
  • Scheduled checkups, so Sherpa re-checks a store every week and opens a fix the moment something breaks.

Built With

Share this project:

Updates

Submission history