Countersign

• 

  Review grid: Claude drafts six PayPal changes; three run inside the mandate, three wait for the seller

• 

  Alex: a hidden fake SYSTEM NOTE is flagged as prompt injection and held for review

• 

  Daniel: $129 refund countersigned, executed in the PayPal sandbox and read back from PayPal

• 

  Priya: wholesale PayPal invoice priced by code from the price list, sent after the seller signs

Inspiration

A one-person online shop answers the same emails every day. It arrived late. It arrived broken. Can you invoice my café? Please pause my box. Where is my parcel? Each one ends in a few clicks in PayPal, and each click is a chance to refund the wrong order, refund twice, or give away too much.

Handing those clicks to an AI agent is the obvious next step, and the tools already exist: an LLM can be given a refund API in a few lines. But an agent that can call refund APIs directly is one convincing email away from paying out money it shouldn't. We wanted the speed of an agent with the safety of a human signature, and a clear rule for which is which.

What it does

Countersign reads a shop's customer messages and turns each one into a single proposed PayPal change: a partial or full refund, an invoice, a subscription pause, or just a reply.

• Claude drafts. Claude Haiku 4.5 reads the message (and any attached photo), looks the customer up through read-only PayPal tools, and submits one structured proposal with the evidence it relied on.
• Policy checks. A deterministic policy engine re-reads the order or subscription in PayPal and checks the proposal: does the order belong to the sender, is there enough refundable balance, does the invoice match the price list, did the agent flag manipulation, and is it inside the seller's mandate.
• The seller countersigns. Small changes inside the seller's mandate (by default: refunds up to $25 and 15% of the order, subscription pauses, reply-only items) run on their own. Everything else waits in an AG Grid review until the seller signs. Signing re-runs every check against fresh PayPal data, executes with an idempotency key, and reads the result back from PayPal.
• Everything is logged. Every proposal, verdict, signature and PayPal result lands in an audit log.

The demo store, Hollow Pine Ceramics, has six messages waiting:

Visitors can also change the mandate, write their own customer message (a live Claude call), and press Reset demo to get fresh sandbox orders.

How we built it

PayPal is the system of record. Every money movement and every fact the policy relies on comes from the PayPal REST API in the sandbox:

• Orders v2: each visitor's demo store is seeded with real captured orders paid with PayPal's public sandbox test card, tagged with custom_id (workspace:customer) so policy can prove ownership. Carrier tracking is attached with /track.
• Payments v2: partial and full capture refunds, with PayPal-Request-Id set to the plan id so a retried signature can never refund twice. Capture lookups give the refundable balance.
• Invoicing v2: draft, send, and read back wholesale invoices.
• Catalog Products, Billing Plans, Subscriptions v1: a monthly "Glaze of the Month" box, suspended on request and read back to confirm SUSPENDED.

After every execution the app reads the object back and shows PayPal's own numbers ("PayPal now shows $129.00 refunded of $129.00"), not its own bookkeeping.

AI is a drafter, not an executor. The agent is a hand-written tool-use loop on the Anthropic TypeScript SDK with Claude Haiku 4.5. Its tools are read-only and scoped to the sender: list_customer_records, get_order, get_subscription, get_price_list. It ends by calling submit_proposal, which is validated with Zod; invalid proposals go back to the model as tool errors. Photos are sent as image blocks. The customer's text is wrapped as untrusted input, and instructions inside it are flagged as prompt_injection. The model never decides whether anything executes or how much. That is lib/policy.ts, plain TypeScript with unit tests.

App. Next.js 16 (App Router), React 19, AG Grid Community for the review grid (status, amount and PayPal ID per proposal). Hosted on Vercel; because serverless instances share no memory or disk, each visitor's store and the rate-limit counters live in a private Vercel Blob store with ETag compare-and-swap writes.

Keeping a public demo affordable. The six scripted messages replay recorded Claude Haiku runs, while every PayPal lookup and execution runs live against the visitor's own sandbox objects. Only messages a visitor writes go to Claude live, with per-visitor, daily and total caps and a rules fallback that the UI labels.

Tests. 40 unit and integration tests (policy, the agent loop with a scripted model, the full scenario on a PayPal simulator, the Blob store), 5 browser tests, and an opt-in Playwright suite that runs all six flows end to end on the real PayPal sandbox and checks the refund, invoice and subscription IDs.

Challenges we ran into

• Sandbox merchant country. Our first sandbox business account could not take card payments (PAYEE_NOT_ENABLED_FOR_CARD_PROCESSING), and Invoicing returned 403 until the app had the Invoicing feature. A US business account and a new app fixed both, and the README now walks through it.
• Seeding without a buyer. Judges need fresh orders on every reset, without logging into a buyer account. Card payment_source orders and card-funded subscriptions made that possible from the server.
• A race only the real sandbox exposed. In development, React ran the first-load effect twice. With the simulator both requests finished instantly; with the real sandbox seeding took about ten seconds, the second cookie overwrote the first, and signing failed with "Plan not found". We load once and added a test that fails on the old behavior.
• Serverless state. The first version kept state in a JSON file and memory. On Vercel, routes can land on different instances, so we moved state to a private Blob store with compare-and-swap writes and retried counters.

Accomplishments that we're proud of

• All six flows run end to end on the real PayPal sandbox, with real refund, invoice and subscription IDs on screen.
• The prompt-injection case is stopped twice: Claude flags it, and even unflagged it could not pay out, because the model has no tool that moves money and policy sends a "never arrived" refund on an order the carrier shows as delivered to the seller for review.
• Signing re-checks PayPal, so a refund made elsewhere between drafting and signing cannot cause an over-refund.
• The seller can read every rule that decided their money, in the UI, next to the evidence.

What we learned

• The useful line is not "AI or no AI" but who decides what. Letting the model draft and letting code plus a signature decide made the product both faster and easier to trust.
• Reading the result back from PayPal after every write is cheap and removes a whole class of "the app says done, PayPal says otherwise" bugs.
• The sandbox behaves differently by merchant country and app features; probing every API call on day one saved us from discovering it during the demo.

What's next for Countersign

• Disputes: draft evidence for PayPal disputes, signed by the seller before submission.
• Shipping: add tracking from carrier emails under the mandate.
• Per-customer and per-day mandate limits, and a weekly digest of everything that ran without a signature.
• Connect a real inbox (email or marketplace messages) instead of the demo inbox.

Built With

• ag-grid
• anthropic
• claude
• claude-haiku
• next.js
• paypal
• paypal-invoicing-api
• paypal-orders-api
• paypal-payments-api
• paypal-subscriptions-api
• playwright
• react
• typescript
• vercel
• vercel-blob
• vitest
• zod