-
-
A listing that hides an instruction only the AI reads
-
The agent’s trace shows it read the hidden note
-
The agent proposes the scam shop
-
The firewall blocks it and voids the PayPal hold
-
An honest purchase passes every check
-
PayPal confirms the capture through verified webhooks
-
Attack Lab: write your own attack
-
Layered defence: rules block, AI only escalates
Inspiration
AI agents are starting to shop and pay for people. They decide by reading web pages, and anyone can write a web page. A seller can hide an instruction in a listing that a human never sees but the agent reads and follows. In my tests, a hidden note fooled the agent 9 times out of 9. Payment platforms already secure the payment itself. Nothing was checking the agent's decision to make it.
What it does
The agent can only propose a purchase. The buyer approves it in PayPal, but PayPal only authorizes (holds) the money. The firewall then compares the agent's claim against trusted data: the user's request and budget, the shop catalog, and the order as PayPal reports it.
- Rules can block. Currency, budget, approved shops, catalog, price and lookalike shop names are checked deterministically.
- AI checks can only escalate. Prompt Guard, two independent LLM judges and an intent check can send a payment to the buyer for review, but they can never approve one.
Each purchase ends in one of three outcomes: allow and capture, hold for the buyer to decide, or block and void. PayPal's verified webhooks confirm the result. An Attack Lab lets anyone write their own hidden instruction and see what happens.
How we built it
- Backend: Python and FastAPI, deployed on Render
- Payments: PayPal REST APIs in sandbox. Orders v2 with intent AUTHORIZE, then capture or void through Payments v2, with idempotency keys and one REST app per merchant
- Webhooks: checked with PayPal's signature verification API
- AI: the agent and judges run on Groq (gpt-oss-120b), with Llama Prompt Guard 2 as a classifier
- Audit: every decision goes into an append-only SQLite log, where database triggers block edits and deletes
- Tests: 454 pytest tests
Challenges we ran into
- Prompt Guard missed natural-language injections, and a single LLM judge missed some attacks. Adding a second judge with a different role took the firewall from 16 of 18 attacks stopped to 18 of 18.
- Voiding another merchant's authorization returned 403 until I gave each shop its own PayPal app.
- Making the system fail safe: if an AI check errors, the payment is held, and if something fails after authorization, the hold is voided.
Accomplishments that we're proud of
The design rule that untrusted text can only make a claim, rules can block, and AI can only escalate. A clever injection can at most send a payment to the buyer for review; it can never approve one. In testing, the firewall stopped all 18 attacks with no false alarms on honest purchases.
What we learned
You can't make the agent itself injection-proof, but you can make sure its output is never trusted on its own. The useful check is comparing the agent's claim against a record the attacker can't touch, which here is PayPal's own order.
What's next for Agent Payment Firewall
A larger blind test set of attacks, support for more currencies and real merchant catalogs, and packaging the firewall as a drop-in check that any agent platform can call before capturing a PayPal payment.
Note: the live demo runs on Render's free plan and sleeps when idle, so the first load can take about a minute.
