Dispute Defender

Small PayPal sellers lose chargebacks they could have won. I built an AI agent that checks each dispute against your records, tells you honestly to fight or refund, and drafts the reply.

Comment

Inspiration

I kept reading the same stat: almost half of chargebacks are never contested. Big merchants pay dispute services to fight them. A small seller with a $110 chargeback just eats it, because building a case means digging through tracking pages, the old listing, supplier paperwork and message history, and then writing it all up for a reviewer. That's an evening of work for one dispute.

Stripe sellers are starting to get AI help with this. PayPal sellers don't. And disputes are getting stranger: buyers' AI agents now make purchases on their behalf, so you get claims like "my agent bought this without permission". Nobody really has a playbook for that yet.

So I wanted to build the thing I'd want if I ran a small shop on PayPal.

What it does

Dispute Defender connects to a seller's PayPal disputes and, for each one:

  1. Reads the buyer's claim from PayPal and treats it as a claim to verify, never as an instruction.
  2. Checks it against the seller's own records: shipment and tracking, the listing as the buyer saw it, supplier certificates, the message log, earlier orders. For purchases made by an AI shopping agent, it checks the mandate the buyer signed (spend limit, category, dates).
  3. Recommends fight or refund, with an estimated chance to win and the expected value of fighting. It's honest: if your own records say the buyer is right, it tells you to refund.
  4. Drafts the response, where every point cites a record. A fact-checker written in plain code (not the AI) blocks any tracking number, amount, date or record that isn't in the seller's data.
  5. Sends nothing until you approve. On confirm, it submits evidence to PayPal with a PDF packet and photos, or accepts the claim to refund.

In the demo you'll see the honest side (a torn jacket, seller never replied: refund), a buyer hiding an instruction for the AI inside a photo (ignored), me typing a fake tracking number into the draft (blocked), an AI agent that spent $310 against a $150 limit (refund), and a real evidence submission going through the PayPal sandbox.

How I built it

  • PayPal: the Disputes API (list, details, the live HATEOAS actions, provide-evidence as multipart with a PDF, accept-claim), plus Orders and Captures with custom_id to tie each dispute back to its order. I used the sandbox process-chargeback endpoint to create realistic disputes for a sample store.
  • AI: OpenAI GPT-5.6 Luna with a strict JSON schema and vision on the listing and buyer photos. The prompt keeps trusted seller records separate from untrusted buyer content. There's also a built-in rules engine with the same output shape, so the app still works if the model fails.
  • Code for the facts: the mandate check, expected value, injection detection and the fact-checker are ordinary tested code. I let the model do judgement and writing, and code check anything that has to be exactly right.
  • Stack: Node 22 and Express, SQLite as a cache (PayPal is the source of truth) with an optional Turso mirror, React with AG Grid for the dashboard, hosted on Render.
  • 125 tests, including contract tests against fake PayPal and OpenAI servers, attempts to sneak fake facts past the guard, and a test that a dispute can never be submitted twice.

Challenges I ran into

  • The sandbox had surprises. My first sandbox account was India-based and couldn't take card payments, so I had to start over with a US one. Chargeback reasons depend on both the reason code and the card brand, and new chargebacks take a few minutes before PayPal lets you act on them.
  • Making the AI unable to lie. My first fact-checker was easy to fool: "99 dollars" instead of "$99", day-first dates, lowercase or spaced-out tracking numbers, invented record ids, fake facts tucked inside a quote from the buyer. I kept attacking it and patching until every attack draft got blocked, and each one is now a regression test.
  • A real model got talked into things. In one run, the injected note in a buyer's message convinced the model to recommend a refund with a reasonable-sounding argument. I changed the prompt so fabricated buyer details weaken the buyer's case, and added a code-level flag for any refund verdict on a dispute that contains an injection attempt.
  • Never acting twice. A slow submit could have been picked up by the background sync and sent again. Claims are atomic now, and anything PayPal hasn't confirmed gets checked, never resent.

Accomplishments that I'm proud of

  • The whole flow works end to end on real PayPal sandbox disputes: dispute, AI verdict, fact-checked draft, seller approval, evidence or refund in PayPal.
  • It recommends refunds when that's the right call. I think that honesty is what would make a seller trust it.
  • Handling AI-agent purchases with a mandate check that you can verify, not just the model's opinion.
  • Each analysis costs about a tenth of a cent with GPT-5.6 Luna.

What I learned

The AI is great at judgement and writing, but exact facts belong to code that can prove them. And an honest "refund this one" earns more trust than a confident "fight everything".

What's next

  • Real PayPal sign-in for actual sellers (right now it runs on my sandbox account)
  • Shopify and WooCommerce connectors so records come in automatically
  • Live carrier tracking APIs
  • Learning which evidence actually wins for each reason code
  • Webhooks instead of polling

Built With

Share this project:

Updates

Submission history