Inspiration
AI agents are starting to shop for people: they hold a saved payment method and check out without asking each time. That's convenient, but it raises a question nobody has a good answer to yet: when the agent buys the wrong thing, who pays? Today it's usually the shopper (stuck with the wrong item) or the small merchant (eating a refund or a chargeback for a mistake they didn't make). The agent, the party that actually made the mistake, pays nothing. I wanted to build the missing accountability layer: a system that makes the party that caused the error pay.
What it does
Recourse sits between a shopper, their AI agent, and the merchant, and uses PayPal to move money according to fault.
- Mandate. The shopper describes what they want in plain language ("Buy a black men's medium cotton hoodie under $40"). Gemini parses it into hard constraints (size, budget) and fuzzy ones (color, style), and the shopper signs it once.
- Agent purchase. A Gemini-powered, tool-calling agent searches the shop's catalog and buys on the shopper's vaulted PayPal account with no further approval. Recourse checks the mandate before every charge.
- Complaint. If something's wrong, the shopper opens a case.
- Ruling. A hybrid adjudicator assigns fault:
| Ruling | Meaning | Money movement |
|---|---|---|
AGENT_ERROR |
Agent bought outside the mandate | Merchant refunds buyer; agent operator is invoiced |
MERCHANT_MISMATCH |
Item doesn't match its listing | Merchant refunds buyer and absorbs the cost |
WITHIN_INTENT |
Item matches the mandate | No refund; the purchase stands |
- Remedy. Refunds and invoices run through PayPal. When the operator pays the invoice, the merchant is made whole ("Recovered from operator"). Any ruling below 70% confidence goes to human review, and no money moves.
An operator view shows invoices, the agent's error scorecard, incoming PayPal webhooks with their signature-verification status, and Gemini usage per model.
What's new compared to what exists today
Agentic payments are moving fast, but almost all of the work so far is about authorizing agents, not about who pays when an authorized agent gets it wrong.
| What exists today | What it covers | The gap |
|---|---|---|
| Card disputes / chargebacks | Buyer vs merchant | Built for two parties. The agent's provider isn't a party, so the merchant usually absorbs the loss. |
| Agent identity & tokens (Visa Intelligent Commerce, Mastercard Agent Pay) | Proving a transaction came from a registered agent | Doesn't reassign liability when an authorized agent simply buys the wrong thing. |
| Signed mandates (Google's AP2) | Evidence of what the user asked for and what the agent bought | Records the evidence, but something still has to judge whether the purchase matched the intent, and move the money. |
| Agent purchase protection programs | Fraudulent agent purchases | Not a purchase that was authorized and simply wrong. |
What Recourse adds:
- An adjudicator for the "authorized but wrong" case. It compares what the agent bought against the signed mandate: rules for hard facts, Gemini for judgment calls.
- The agent operator as a liable third party. When the agent is at fault, the operator gets a PayPal invoice, and the merchant is made whole when it's paid. Today the agent's side typically pays nothing.
- Remedies in minutes, at the payment layer. Refunds and invoices run through PayPal right after the ruling, instead of a weeks-long chargeback.
- Fair in both directions. It also protects merchants: if the agent did exactly what was asked, the claim is denied with an explanation.
Recourse doesn't compete with mandate protocols like AP2. It's the missing piece that consumes that kind of evidence, decides fault, and settles it.
How I built it
- Stack: Next.js, TypeScript, Tailwind, Drizzle ORM, Postgres, hosted on Render.
- PayPal (sandbox): Orders v2 and Vault for agent-initiated charges with no buyer present, Payments v2 for refunds, Invoicing v2 to bill the agent operator, and Webhooks (signature-verified) to track refund and invoice state.
- Gemini (gemini-2.5-flash) has three jobs:
- parses the shopper's request into a structured mandate,
- drives the shopping agent through tool calls (
search_catalog, thenpurchase), - judges the fuzzy parts of disputes (color synonyms, style, compatibility).
- Hybrid adjudicator: deterministic rules decide hard constraints (price, size, quantity, delivery, numeric specs). Gemini judges the rest. The LLM can never override a failed hard check.
- Tamper-evident intent: mandates are signed with a server key, so the record of what the shopper asked for can't be quietly edited later (prototype-grade).
- Evaluation: 30 labelled cases, 10 for tuning and 20 held out for reporting.
Challenges I ran into
- Deciding what an LLM should be trusted with. Letting a model trigger money movement is risky. Splitting rules from LLM judgment, and adding the confidence threshold for human review, took several iterations.
- Free-tier quota limits. Gemini's free tier allows 20 requests per model per day. Outputs are cached, models fail over in order, and when quota runs out the app falls back to rules, labels the result, and routes it to human review without moving money. Two demo cases replay saved Gemini outputs (labelled in the UI), so the demo always works.
- Charging with no buyer present. Getting vaulted, agent-initiated payments working end to end in the PayPal sandbox took careful handling of the Vault and Orders flows.
- Closing the loop on recovery. The merchant panel only shows "Recovered" once PayPal's invoice-paid webhook actually arrives, not when the invoice is sent.
- Honest evaluation. On the held-out set, Hybrid and LLM-only both scored 20/20, while Rules-only scored 12/20 and flagged 19 for review. Adding rules didn't improve accuracy, so I had to be clear about why I still shipped hybrid: hard violations are decided deterministically, and they still resolve when the LLM is unavailable.
Accomplishments that I'm proud of
- A complete loop from intent → agent purchase → ruling → refund → invoice → recovery, built on five PayPal APIs.
- Fault that includes the agent operator as a third party, not just buyer versus merchant.
- A design where the AI makes the judgment calls but can't override hard facts, and uncertain cases default to a human.
- Being upfront in the product itself: replayed decisions, rules-only fallbacks, and injected demo errors are all labelled in the UI.
What I learned
- Fault in agentic commerce really comes down to intent: you can't decide who was wrong without a signed, verifiable record of what the shopper asked for.
- Hybrid AI systems are mostly about deciding where the LLM isn't allowed to decide.
- A 20-case eval is enough to catch major problems, but too small to prove much. Being clear about its limits builds more trust than an inflated number.
What's next for Recourse
- Plug into real agent frameworks and merchant catalogs instead of a simulated shop.
- A larger, independently labelled eval set, including adversarial cases.
- Integration with PayPal's dispute flow, so rulings can head off chargebacks instead of only refunding.
- Proper key management and verifiable signatures for mandates.
- Operator reputation scores based on each agent's error rate.
Prototype note: PayPal runs in sandbox only, the shopping agent and catalog are simulated, the demo's agent error is injected on request and labelled, and scorecard and chargeback figures are demo data or industry estimates.
Built With
- claude
- docker
- drizzle-orm
- gemini-2.5-flash
- google-gemini
- llm
- nextjs
- node.js
- paypal
- paypal-invoicing-api
- paypal-orders-api
- paypal-payments-api
- paypal-vault
- paypal-webhooks
- postgresql
- react
- render
- tailwindcss
- typescript