Chargeback Copilot

An AI disputes analyst for small PayPal merchants: it investigates every dispute, tells you which ones are worth fighting, drafts the evidence and submits it once you approve.

Comment

Inspiration

Small online sellers lose real money to disputes, and not only from fraud. Every "item not received" or "not as described" claim means digging through carrier tracking, old emails and order notes, then writing a response before a deadline. Most small merchants don't have that time, so they refund everything or miss the deadline. Fighting a case they can't win costs them too: fees, plus hours of work. We wanted a tool that does the investigation and gives an honest answer: fight this one, settle that one, refund this one.

What it does

  • Dispute queue: pulls every open dispute from the PayPal Disputes API into one list sorted by deadline, showing the amount at risk, what's due next, and the expected recovery if you follow the recommendations.
  • AI investigation: an LLM agent built on the PayPal Agent Toolkit reads the dispute, checks shipment tracking through PayPal's Tracking API, looks for related transactions, and searches the merchant's own records: order notes, support messages, refunds, listing text, and anything the merchant pastes into the case. Each step appears live in an analysis log.
  • Scoring: the agent passes only verified facts to a deterministic scoring rubric. The rubric produces a win likelihood and the expected money kept under each option (submit evidence, offer a partial refund, or refund) and recommends whichever keeps the most. The recommendation is explainable and consistent, not a model's guess.
  • Drafted response: the agent writes the evidence statement, citing only tracking numbers, refund IDs and policy text it actually found, and picks the PayPal evidence type.
  • Human approval: the merchant reviews and edits, then confirms. Only then does the app call the Disputes API to provide evidence, make an offer, or accept the claim. The AI agent only gets read-only PayPal tools.

We tested this end to end in the PayPal sandbox. A real $92 dispute was first scored at 5% with no proof on file. After the merchant added a courier receipt, it rose to 90%, and the submitted evidence moved the dispute from waiting on the seller to waiting on the buyer.

How we built it

  • Next.js 15 (TypeScript) with Tailwind, and AG Grid for the dispute queue.
  • PayPal Agent Toolkit (@paypal/agent-toolkit) for the agent's tools: get dispute, shipment tracking, transactions.
  • PayPal Disputes REST API for list and get, provide-evidence, make-offer and accept-claim.
  • Vercel AI SDK with tool calling, running Groq (openai/gpt-oss-120b). It also works with Anthropic, OpenAI or Gemini keys.
  • A deterministic rules engine fallback. If the AI model fails or no key is set, the same tools and rubric still produce a recommendation, so the app never stops working.
  • Deployed on Render, with a keep-alive so the free instance stays responsive.

Challenges we ran into

  • PayPal evidence types depend on the dispute. An item-not-received inquiry rejected "proof of delivery signature" with EVIDENCE_TYPE_IS_NOT_ALLOWED. We added a fallback that tries the AI's choice, then the evidence types PayPal accepts for that case.
  • Multipart evidence uploads. PayPal rejected our first submissions as malformed until we built the multipart request exactly as the API expects.
  • Free-tier AI limits. Groq's free tier allows 8,000 tokens per minute, and a multi-step agent quickly runs past that. We cut what the agent sends (compact dispute data, trimmed tool results) and added backoff, so one full investigation fits within a minute.
  • Open models sending messy tool calls. The model sometimes sent empty or mislabelled scoring fields, which Groq rejected. We made the scoring tool tolerant and normalise its input ourselves.
  • Sandbox setup from India. India sandbox accounts can't make domestic person-to-person payments, so we used a US merchant account to create real test disputes.

Accomplishments that we're proud of

  • A real dispute, analysed by AI, answered through PayPal's API, with the status change verified in PayPal's own system.
  • It doesn't fight everything. The tool recommends partial refunds or refunds when that keeps more money.
  • Recommendations are explainable: every score traces back to specific facts and costs.

What we learned

  • How PayPal's dispute lifecycle works in practice: inquiries, seller-response windows, evidence types, and how the sandbox behaves.
  • That separating "AI gathers facts" from "rules decide" makes an AI system far easier to trust and debug.

What's next for Chargeback Copilot

  • Pull merchant records automatically from store platforms (Shopify, WooCommerce) and helpdesks, instead of a local file plus pasted notes.
  • Webhooks for new disputes, so analysis starts the moment a case opens.
  • Learn from outcomes: calibrate the win-likelihood rubric using real dispute results.
  • Upload documents (receipts, photos) as evidence attachments.

Built With

Share this project:

Updates

Submission history