Inspiration

Give an AI agent a PayPal key, and the first store page it reads can talk to it. In my tests, a real model was talked into a $1,620.64 order shipped to an attacker's PO box, on the PayPal sandbox. Guardrails are easy to say and hard to price: every defense also refuses honest buyers, asks the owner more questions, or holds a seller's money. I wanted one place to see which defense is worth its cost, and to check that it actually changes what PayPal does.

What it does

  • A real model, live. On the home page, Llama 3.2 3B shops three listings on my server. One listing hides an instruction to ship somewhere else. Its exact order then goes through the checkout twice: with no checks (the PayPal sandbox captures it) and with the recommended rulebook (stopped before PayPal). Some runs it is fooled, some not; the page shows what happened.
  • Send your own agent in. The shop is open over HTTP: POST /api/shop starts a session, your agent reads the listings and places an order, and the site draws every move at a watch link. No account, no clone. A short Python script for Ollama or OpenAI-style models is served from the site.
  • Test before you release. 16 cases (listings that fooled real models, honest listings the agent must still buy, checkout calls, replayed days). Plain code gives a verdict, a fix per failure and a Markdown report, and exits 1 on a breach so it runs in CI.
  • Price each defense. Thirty simulated days with 630 shoppers, stores and six attack families. Each rulebook is measured on value kept, money stolen, honest buyers refused and seller money held. You set what each costs your business, and the recommendation changes with your prices.
  • Enforce it at PayPal. The same rulebook runs in POST /api/checkout (Orders v2: create, authorize, capture, void, refund) and wraps PayPal's Agent Toolkit in one file, so create_order and pay_order pass through it.

How I built it

A TypeScript monorepo: a deterministic engine that runs in a Web Worker or Node, a Fastify server with the PayPal adapter and a durable payment journal, and a React front end with a PixiJS city and AG Grid ledgers. Every pass, fail, allow and refuse is decided by plain code; models are only the agents under test.

  • PayPal: Orders v2 on the sandbox, holds as authorizations that are captured after delivery or voided, PayPal-Request-Id on retries, read-back reconciliation by id, a verified webhook (a forged one gets 401), and @paypal/agent-toolkit with a model making the payment calls.
  • Crash safety: the capture intent is written before the capture leaves. A test kills the process right after a capture; recovery reads the order back from PayPal and charges once. A client without the journal charges twice.
  • Method: rulebooks were frozen and hashed before each benchmark round, and the rule for picking a winner was written first.
  • Models: Llama 3.2 3B, Llama 3.1 8B, Mistral 7B and Qwen 2.5 7B on Ollama, Qwen3.5 9B on Together, Gemma 4 26B on the Gemini API. All paid model time came to $0.112.

Challenges I ran into

  • An outside review found I had counted scanned orders as delivered, and blamed one rulebook for owner step-ups another rule caused. I fixed both and kept the wrong results in the record.
  • My first repair for empty-box stores did not work. Only the strict hold stops them.
  • Small models are inconsistent: the live run is fooled most of the time, not every time, so the page reports each run as it happens instead of a staged result.

Accomplishments that I'm proud of

  • With real models (3,072 shopping trips under a protocol frozen first), the checkout gate cut the loss per attacked trip from $85.66 to $36.52, and honest purchases held at 111 of 128.
  • An agent I wrote outside the project (Python and Qwen 2.5 7B) shipped to the attacker's address. A short guard fixed it, and it still bought every honest listing.
  • On the sandbox, a store that stops shipping kept $574.82 with no hold and $64.48 under the strict hold, while an honest store with the same record got all its money, later.

What I learned

The winning defense was not the cleverest one. It was the owner's mandate (budget, quantity, address), checked in code before money moves. Which rulebook wins after that depends on your prices, not mine.

What's next for Sandbox City

More outside agents through the open shop, a one-line install for CI, and real fee and dispute data in place of my stated assumptions.

Limits

The days are simulated and I wrote the attacks. Fees and approval rates are assumptions. Everything ran on the PayPal sandbox; no real money moved.

Built With

Share this project:

Updates

Submission history