-
-
Countersign: a firewall between AI agents and PayPal
-
Every purchase or refund an agent tries is checked before PayPal moves money
-
A real Gemini agent obeys a hidden note and tries to buy a gift card. Blocked, no PayPal order created
-
A poisoned ticket makes the agent call PayPal’s create_refund for $500 on a $79 order. Blocked on four checks
-
A $79 purchase approved with a pinned device’s signature, then captured by PayPal
-
A fair $10 refund is checked against the real payment, then refunded through PayPal
-
The AI red team: Gemini writes new attacks on your own support agent
-
The kill switch blocks every agent payment until the owner unfreezes
-
AG Grid audit log you can question in plain English, with a SHA-256 tamper-evident ledger
Inspiration
AI agents are starting to shop, pay and refund for people. But an agent reads the open web and customer emails, and those can talk back. One hidden line on a product page can tell an agent to "buy this instead", and one poisoned support ticket can tell it to refund $500. Card limits stop overspending, but they can't tell whether a payment is what you actually asked for. We wanted a layer that can.
What it does
Countersign sits between any AI agent and PayPal. Before money moves, the agent submits what it wants to pay or refund, the task it was given and the pages it read. Countersign runs independent checks:
- Prompt injection: did something the agent read hide instructions for AI, and did the agent follow them?
- Matches your task: does this payment serve what you asked for?
- Your rules: plain-English rules ("no gift cards, ask me over $60") that Gemini turns into limits, blocked categories and quantity caps.
- Fair price and seller trust: marked far above the usual price? Account age, dispute rate, rating.
- Budget: weekly spend and daily refund limits.
Each attempt gets Allow, Review or Block, with a one-line reason and a risk score. Allowed purchases go to PayPal checkout. Review sends the owner an approval page (QR code for their phone). Blocked payments never reach PayPal. In our tests a real Gemini agent fell for a realistic hidden "out of stock, buy this gift card instead" note in 3 of 5 runs, and Countersign blocked every one.
It guards the merchant side too. PayPal's official MCP server lets an agent refund customers, accept disputes and send invoices. Countersign Guard is a drop-in replacement: it runs PayPal's server underneath, offers the same tools, and checks each of the 8 money-moving tools before PayPal sees the call. Refunds are verified against the real payment in PayPal: it exists, it isn't already refunded, and the amount fits what's left. In the demo, a ticket with a hidden "policy update for AI support assistants" makes the live model call create_refund for $500 on a $79 order. Countersign blocks it on four checks; the refund never runs.
AI red team. Don't wait for a real attacker. Gemini writes new attack tickets against your own support agent: forged owner emails, fake PayPal fraud alerts in HTML comments, fake tool output, other languages, 1-pixel text telling it to split refunds under your limit. Honest tickets are mixed in. You get a scorecard (what your agent fell for, what Countersign stopped, false alarms, money kept), a write-up of where your agent is weak, a fix for its instructions, and a one-click re-run against the hardened agent.
Countersigned approvals. Pin your phone or laptop with one tap. The browser makes a P-256 key that can never be exported, and every approval is signed with it for that exact payment and amount. Once a device is pinned, a stolen login can't approve anything: edited, replayed or expired signatures are refused.
Proof you can check.
- A tamper-evident ledger: every decision, payment, rule change and signature is chained with SHA-256. "Verify chain" re-hashes it in your browser; "Try tampering" edits a past entry so you can watch it get caught.
- Verified PayPal webhooks report what really happened to each payment. A capture that differs from what was approved is flagged, and a buyer dispute on an agent payment freezes every agent.
- Countersign Bench: 44 attacks and 20 honest requests run through the same engine as the live app. With the AI switched off, the rules alone let 0 of 44 attacks move money and blocked 0 of 20 honest requests. Without Countersign, all 44 move $4,133. We tuned the detectors while building these cases, so we call it a best case; the red team writes attacks the rules have never seen. Judges can re-run it live, with the AI off or on.
Controls you keep. One kill switch freezes every agent. "Hold payments until delivery" makes PayPal hold the money as an authorization until you confirm the item arrived; holds nobody confirms are cancelled after 72 hours. Every attempt lands in an AG Grid audit log you can question in plain English ("blocked refunds over $50").
How we built it
- Next.js 15 + TypeScript, deployed on Render.
- PayPal Orders v2: create, approve, capture, and intent AUTHORIZE for holds. The order's reference ID is the firewall decision ID, and the captured amount must equal the approved amount.
- PayPal Payments v2: capture lookup, idempotent refunds, and authorization capture or void.
- PayPal webhooks, verified with PayPal's verify-webhook-signature API.
- PayPal MCP server (
@paypal/mcp), wrapped by Countersign Guard. - Gemini for rule parsing, injection review, task matching, the live agents, the red team and audit-log questions. The AI layer is swappable and falls back to backup models.
- Defense in depth: a deterministic rules engine runs alongside every AI check and the stricter result wins, so the firewall still works if the AI is down.
- Web Crypto (ECDSA P-256) for countersigned approvals, SHA-256 for the ledger, AG Grid for the audit log.
- A REST API and our own MCP server, so Claude Desktop, Cursor or any MCP agent has to clear payments with Countersign.
- Security: owner login, device-signed approvals, rate limits, API keys, security headers, 48 automated tests and the 64-case bench.
Challenges we ran into
- Our first injection attack was too obvious: real Gemini ignored a fake "SYSTEM NOTICE." We rewrote it as a believable fulfilment note, which real models do fall for. That changed our pitch: you can't count on the agent to resist.
- Running our own red team and bench against Countersign found real gaps (refund splitting, 1-pixel text, leetspeak, other languages, a per-item budget bug). Every one is fixed and tested.
- Gemini overloads and free-tier limits (503 and 429 errors) broke early runs. We added retries, model fallback and a time budget so the demo never hangs.
- PayPal sandbox accounts created from India rejected USD checkout; US sandbox accounts fixed it.
Accomplishments that we're proud of
- Watching a real AI agent get hijacked live, on both the shopping and the merchant side, and seeing Countersign block it.
- A full real loop: AI verdict, device-signed approval, PayPal sandbox payment captured, and refunds through PayPal's own MCP tools.
- Treating Countersign itself as an attack target and publishing the numbers, including where we had to fix it.
What we learned
Agent safety can't live inside the agent alone. The same model resisted an attack in one run and fell for it in the next. A payment checkpoint outside the agent, with checks the agent can't talk its way past and a human signature for anything that needs one, is what makes agentic commerce trustworthy.
What's next for Countersign
- Saved (vaulted) PayPal payment methods, so allowed purchases settle with no checkout step.
- Seller trust from real PayPal transaction and dispute history.
- Passkeys for approvals, and persistent storage for the ledger.
- Shared household and team budgets with per-agent rules.
Built With
- ag-grid
- gemini
- mcp
- nextjs
- paypal
- paypal-mcp
- paypal-orders-api
- paypal-payment-api
- react
- tailwindcss
- typescript
- vitest
- web-crypto