-
-
A $9.00 sandbox refund under the $25 limit goes out on its own, with PayPal's reference on screen. Test money only.
-
A $40.00 refund is over the limit, so it waits for the merchant's fingerprint. If they do nothing, no money moves.
-
A fake system message in the demo: in this run the checks caught it, nothing was sent, and every step is shown.
Inspiration
This is a hackathon concept and a learning project, sandbox only. It is not production software and has had no independent security review. We are not claiming it cannot be beaten: our own testing beat it (see What we learned).
AI assistants are starting to handle refunds and disputes for small merchants. PayPal's agent tooling can already call money-out actions such as refunds, but whether an agent may use a tool is an on or off choice per tool: there is no amount limit, no daily limit and no approval step in between. That leaves a merchant two bad options: let an AI move money without limits, or not automate at all.
The risk is not hypothetical. Customer messages are untrusted text, and an AI that reads them can be talked into things: OWASP lists goal hijack and tool misuse as the top risks for agentic AI (Agentic Top 10, Dec 2025), and a tribunal held Air Canada liable for what its chatbot told a customer about refunds (Moffatt v. Air Canada, 2024 BCCRT 149). Banks solved this decades ago with maker-checker controls: the person who prepares a payment is never the one who approves it. We applied the same control to AI.
What it does
AgentCoSign sits between an AI refund assistant and PayPal.
- The AI proposes, never decides. A refund assistant reads the customer's message and proposes a refund. It holds no PayPal credentials and cannot send anything.
- Plain rules decide. A deterministic policy engine (18 rules, no model inside) checks every proposal against facts it reads from PayPal itself: what was captured, what was already refunded, open disputes, currency and account. Small, clean refunds within the merchant's automatic limit go out on their own. Anything larger, unusual or disputed is sent to the merchant. Anything that could not be checked is refused. The strictest outcome always wins.
- Independent AI checks. Before the rules decide, three separate checks read the customer's own words: a prompt guard scores attack wording, a screening model looks for manipulation such as a message pretending to be the shop or PayPal, and a separate model reads what the customer actually asked for (order, amount, reason, and whether they want the money sent somewhere else). A refund goes out alone only when the assistant's proposal, the customer's own words and PayPal's facts all agree.
- The merchant co-signs on their phone. When approval is needed, the merchant sees a plain-language card (who, how much, why it needs them, what the AI read, labelled as AI) and approves with a passkey: their fingerprint, face or PIN signs that exact refund. Doing nothing sends nothing.
- A signed record keeps it. Every decision, approval and PayPal answer is written to a hash-chained, HMAC-sealed record, so a changed or deleted entry is detectable.
- An emergency stop. One button stops every money action; only the merchant's fingerprint turns refunds back on.
For judges: see it and look inside it. The hosted demo gives every visitor their own private workspace. Set it up with your fingerprint or PIN, press Try a scenario (a genuine refund, one over the limit, a "charged twice" claim, a fake system message, a request to send the money to a friend, or your own message), and each creates a real PayPal sandbox order. Switch on Behind the scenes to watch, live beside the app, every step the server takes: each PayPal call with PayPal's own ids, each model's answer, the rules' decision in plain words, the signed record entries, and an outcome block saying exactly what happened to the money.
Why it matters
- For merchants: a shop that lets AI answer customers today must choose between giving it unlimited refund power and doing every refund by hand. AgentCoSign is a third option: small, clean refunds go out on their own within a set limit, and anything else waits for the owner's fingerprint. The most an AI mistake can cost, per refund and per day, is a number decided in advance.
- For customers: a genuine small refund does not have to wait for a person. Only a refund that needs a look waits, and the customer is told when to expect an update.
- For developers building on PayPal: the same control can sit in front of any AI action that moves money, such as refunds, accepting a dispute or payouts. Builders get amount limits, approvals and a signed record without having to build them from scratch.
- For trust: every decision is kept in a tamper-evident record, so "why did this refund go out?" has an answer.
This is a concept tested only in the sandbox; the impact case has not yet been checked with merchants.
How we built it
- Backend: Python 3.12, FastAPI, Pydantic v2 with strict validation at every edge. SQLite in WAL mode for the ledger, with budget reserved atomically, so two requests at the same moment are designed not to spend the same limit twice.
- PayPal: our own REST client against the PayPal sandbox (see Tools below), with a PayPal-Request-Id on every write so a retried refund is designed not to be paid twice, and a fresh read of the order immediately before every refund is sent.
- AI: four model roles on Groq, each a different job and, where it matters, a different model family, with Zero Data Retention switched on for customer text. Customer messages are masked (cards, security codes, emails, phone numbers, bank and ID numbers) before any model sees them. Model output is schema-validated data, never executed and never shown as the merchant's words.
- Approver app: a vanilla JavaScript progressive web app with WebAuthn passkeys; each approval signature is bound to a digest of the exact refund.
- Engineering discipline: 1,348 offline tests with 95.9% coverage, a 113-row policy scenario table, property tests for bounded loss, mutation checks on new guards, secret scanning, and a configuration check that stops the app from starting on invalid settings.
How we used AI to build it
AgentCoSign was built by one person, Vikas Sahani, with Claude Code (Anthropic, on the Claude Pro plan) as the engineering partner. We are stating plainly how that worked, because the split is the same idea as the product itself: the AI proposes, a person decides.
The owner was the decision maker and business problem solver. Vikas defined the problem (unbounded AI money movement for small merchants) and the product: which rules exist and their limits, when a refund may go out alone and when the merchant must be asked, how the merchant's screens should read, and what "safe" means in money terms. Every specification was approved by Vikas before it was built. Every action with real consequences stayed with Vikas: creating and holding the PayPal and Groq accounts and keys, turning on Zero Data Retention, registering the PayPal webhook, deployment and repository visibility, and every passkey approval in testing, signed with Vikas's own fingerprint or PIN.
Claude Code was the engineering partner. Under that direction it researched options and sources, wrote and revised code, tests and documentation drafts, ran the test suites, drove the browser during user-acceptance runs while the owner signed, and reported findings, including its own mistakes. The file holding the PayPal and model keys was blocked from it, though test scripts it ran did use those keys. It could not approve anything.
AI-written work was held to the same standard as the product. Every change had to pass the offline test suite, coverage and lint gates, secret scanning and a project checklist on each commit; every new safety check was mutation-tested (broken on purpose to prove its test fails); and live runs against the PayPal sandbox were checked by the owner.
Tools used and how
- PayPal Orders API v2: Create and capture sandbox test orders (scenarios); read an order's captures, refunds and payee as the evidence the rules decide on
- PayPal Payments API v2 (refunds): Send refunds, idempotent through PayPal-Request-Id; read refund status
- PayPal Customer Disputes API v1: Check an order for open disputes before any refund (a disputed order always goes to the merchant)
- PayPal Webhooks: Signature-verified events (capture refunded or reversed, refund failed, dispute created, updated, resolved) to catch changes made outside the app
- PayPal OAuth 2.0: Client-credentials tokens, refreshed on expiry
- PayPal Developer Dashboard and sandbox accounts: Sandbox business and buyer accounts, published test cards
- Groq,
openai/gpt-oss-120b: The refund assistant that proposes (falls back toqwen/qwen3.8-27b) - Groq,
openai/gpt-oss-20b: Safety screening of customer messages (falls back toopenai/gpt-oss-safeguard-20b) - Groq,
qwen/qwen3.8-27b: Reads the customer's own words, separately from the assistant (falls back toopenai/gpt-oss-20b) - Groq,
meta-llama/llama-prompt-guard-2-86m: Scores attack wording from 0 to 1 - WebAuthn (browser passkeys): The merchant's co-signature
- Render: Hosting the public demo (free tier)
- Claude Code (Anthropic, Claude Pro plan): Engineering partner throughout the build, steered by the owner (see "How we used AI to build it")
- ChatGPT (OpenAI): Generated the README banner and other image assets
- Figma Make: Generated the clickable merchant app concept (a design concept, not part of the working product)
Challenges we ran into
- Making "AI-safe" measurable. We wrote 60 labelled customer threads (genuine requests and attacks, grounded in real refund-abuse patterns) and measured every attack twice: as the assistant actually behaved, and as if the attacker had fully taken it over.
- Truthful money states. A refund is "sent" only when PayPal says completed. If PayPal's answer is lost, the app says so and checks PayPal before any retry, never "nothing was sent".
- Splitting a big refund into small automatic ones. We found that three small refunds could have slipped under the automatic limit, and added a rule that caps automatic refunds per order and per customer per day.
- Showing the backend without leaking it. The behind-the-scenes panel shows real ids and timings but never bodies, secrets, sign-in tokens or unmasked customer text, and each visitor sees only their own workspace.
Accomplishments that we're proud of
- A small test set of 60 synthetic threads written by the author: 0 of 40 attacks moved money without the merchant with the assistant assumed taken over, and 0 of 20 genuine customers were flagged. That set did not include the attack described in What we learned. (Live run, Oct 25; report in the repo.)
- 15 real sandbox refunds, sent automatically and after a fingerprint approval, each read back from PayPal as COMPLETED.
- Twelve attack styles typed live on a real phone against the hosted demo, with the results published in the repo, including the cases where money moved.
- A merchant app written for a busy, non-technical shop owner on a phone: plain words, one decision per screen, no ticking timers.
What we learned
The AI checks are a useful layer, not the safety net. On held-out attacks screening alone caught 68% (13 of 19). The deterministic limits and the merchant's approval reduced what a miss could cost.
The most useful lesson came last. With all automated tests passing, a typed message aimed a refund at another customer's order, and $20.00 went out on its own. The assistant, the reading and PayPal's facts all agreed, because all three followed the order the message named: checks that read the same attacker-written text are not independent. We fixed it by tying each message to its order at intake, and the write-up is in the repo. We expect there are more problems like it, which is why the demo invites people to try.
What's next for AgentCoSign
- An idea for a merchant app, untested at any scale: a merchant connects their own business account with consent through PayPal's partner onboarding, and their phone becomes the authenticator for every payment leaving the account, AI-proposed or staff-proposed, with a real-time push and a passkey signature for that exact payment. Facts stay read live from PayPal's APIs. Needs partner onboarding and production review, per-merchant key isolation, push notifications and compliance work (not built).
- Talking to merchants: the impact case is reasoned so far, not yet validated with interviews.
- Payouts and dispute acceptance behind the same controls.
- Reconciliation that flags a dispute opened after a refund.
- A merchant-chosen policy per shop, and native mobile approval.
Built With
- claude
- fastapi
- gpt-oss
- groq
- javascript
- llama-prompt-guard
- passkeys
- paypal
- paypal-disputes-api
- paypal-orders-api
- paypal-payments-api
- paypal-webhooks
- pwa
- pydantic
- python
- qwen
- render
- sqlite
- webauthn