Inspiration

AI agents are about to do the shopping. Everyone is building the agents. Nobody is asking whether the stores survive them.

So we asked. Then we built the thing that answers it.

What it does

Point PayPal Swarm at a PayPal sandbox store. It releases a crowd of LLM shoppers with budgets, intents and limited patience, a merchant agent built on the PayPal Agent Toolkit, and a chaos agent that injects declines, replayed approvals, oversized refunds, chargebacks and lost parcels.

You watch it live in the Arena. The Report scores the store 0–100 and ties every finding to the exact PayPal ids behind it, with a one-line fix. Press Re-run with fix and watch the score move. The Studio (AG Studio) lets an AI analyst interrogate the event log.

The score is not a vibe:

$$\text{score} = \sum_{d} w_d \cdot \max\Big(0,\ 100 - \sum_{r \in d} \min(40,\ p_r)\Big)$$

Four dimensions \(d\): Discoverability 20 %, Transactability 35 %, Fulfilment 25 %, Resilience 20 %. Ten rules \(r\), each deducting \(p_r\) points by severity (critical 25, high 15, medium 8, low 3).

How we built it

As a team of two, in TypeScript. One append-only event log is the source of truth, the Arena streams it, the Report is a pure function over it, the Studio tables are projections of it.

  • PayPal Agent Toolkit for every merchant action and all shopper browsing, behind per-agent allow-lists
  • PayPal Server SDK and sandbox REST for card approvals, idempotent captures and real sandbox chargebacks
  • Vercel AI SDK + OpenRouter for the agents, Hono for the API, Next.js and AG Studio for the UI, Postgres for storage
  • A simulated clock (1 real minute = 1 sim hour), so a 24-hour SLA breach fires inside a 5-minute run

Challenges

  • PayPal says 200 when the card was declined. The capture status is buried in the body. Our naive merchant shipped anyway. That became our one critical rule.
  • No API approves an order as a wallet buyer. We approve by sandbox test card instead.
  • Hidden reasoning ate the token budget. 30 of 78 shopper turns in one Black Friday run came back empty until we switched reasoning off.
  • Our own cost meter lied. It under-reported spend by 2–8×. We found it, fixed it, and re-measured.

What we learned

We ran the same scenarios on eight models, from $0.002 a run to $0.74. The verdict barely moved: the unfixed store scored 82–91 on every one, the fixed store 93–100.

The score follows the merchant, not the model. A frontier model does not rescue a store that never ships. Fix the process, and a one-cent run on Haiku 5.5 scores 100.

Your store is not ready. Now you can prove it for a cent.

Built With

  • ag-grid
  • ag-studio
  • anthropic
  • claude
  • drizzle
  • github-actions
  • hono
  • next.js
  • node.js
  • openapi
  • openrouter
  • paypal
  • paypal-agent-toolkit
  • paypal-server-sdk
  • postgresql
  • react
  • render
  • server-sent-events
  • tailwindcss
  • typescript
  • vercel-ai-sdk
  • vitest
  • webhooks
  • zod
Share this project:

Updates

Submission history