Inspiration

When a buyer disputes a payment or a card issuer files a chargeback, a small seller has days to answer with the right evidence. Most refund everything or send a generic note. Each chargeback costs U.S. merchants about $128 in handling and third-party fees on top of the lost sale (Mastercard/Javelin, 2025). A lot of the proof the seller needs (what was ordered, the tracking number) is already inside PayPal.

What it does

Casefile is a one-screen dispute desk running against the real PayPal sandbox. The part we think is new: after it files, it reads PayPal’s stored copy back and proves it is byte-for-byte what the seller approved (notes, tracking, and the PDF downloaded from PayPal’s document store, compared by SHA-256), and shows a receipt with PayPal’s debug id.

Why: with an AI drafting, “review, then send” is not “send what I reviewed”. On 8 live sandbox disputes, regenerating the notes at send time gave different text from the reviewed draft in 7 of 8. With the approval bound to the hash of the exact call and checked against PayPal’s copy, 9 of 9 real filings matched (7 supporting-info with PDF, 1 provide-evidence with tracking, 1 accept-claim; 3 filed by clicking in the web UI).

  1. What PayPal says. It reads the dispute through the Disputes API: reason, requested evidence and the actions PayPal offers right now (HATEOAS links).
  2. Evidence. It follows the disputed capture (Payments v2) to the order (Orders v2) to get line items, SKUs and the tracking number added with Orders v2 /track, then adds the shop’s own records (listing, shipping log, refunds). Every item gets an id, E1..En. On 8 live sandbox disputes, the records PayPal holds took coverage of the requested evidence from 0/8 to 8/8.
  3. Decision. Fixed rules choose FIGHT, REFUND or NO_ACTION, and flag the cases the seller should lose (never shipped, duplicate charge, refund owed, short-packed, amount below handling cost). The LLM never decides money.
  4. Notes. Groq’s free tier (openai/gpt-oss-120b) writes the notes, citing evidence ids. A deterministic verifier checks every sentence: numbers, dates, tracking and refund ids must be in the cited evidence; no contact data; no refund promise in a FIGHT; no “address matched” or “delivered/received” claim without a record that says so. Rejected drafts go back to the model once with the errors, then fall back to a template. The screen shows rejected drafts and why.
  5. Approve, file, verify. The desk shows the exact PayPal call and sends nothing until the seller approves a token bound to that call (and to the SHA-256 of the attached PDF): provide-evidence with tracking info, provide-supporting-info with a one-page casefile PDF while PayPal reviews a chargeback, or accept-claim. Then receipt.py re-reads the dispute, finds the new seller entry, downloads the stored PDF and compares: green receipt on a match, red with the differing field otherwise.

Live demo: https://casefile-igor-eduardo.vercel.app (also https://casefile-kohl.vercel.app). The landing, recorded receipts, and queue are public (no Vercel login). Write actions that touch the PayPal sandbox are unlocked only from the judges-only testing-instructions field, not in this public text. After unlock, press Open a real sandbox chargeback and answer it: a card payment, tracking and a chargeback are created in the PayPal sandbox, and about two minutes later the dispute is ready to answer and verify.

How I built it

Python standard library only (no framework, no SDK). Modules: paypal.py (sandbox-only REST client that keeps the PayPal-Debug-Id of every response), records.py, evidence.py, decide.py, writer.py, verify.py, document.py (PDF), desk.py, receipt.py, sandbox_seed.py, cli.py, web.py. scripts/sandbox_e2e.py creates demo-shop card payments, adds tracking, files sandbox chargebacks (process-chargeback) and runs the full loop unattended. The hosted demo runs as a Python function on Vercel Hobby.

What ran in the PayPal sandbox (Nov 11–12, 2026)

  • 27 card chargebacks on demo-shop purchases (plus a later hosted-demo run, dispute PP-R-CLN-10199588, which also returned a green receipt, PayPal debug id f836639ebd9ae).
  • PayPal accepted provide-supporting-info with the casefile PDF on 8 disputes in the Nov 11 set; the PDF downloaded back from PayPal’s document store is byte-identical to the one sent.
  • PayPal accepted provide-evidence on 2 disputes (status moved to UNDER_REVIEW; PayPal lists PROOF_OF_FULFILLMENT with the USPS tracking number) and accept-claim on 1 (ACCEPT_CLAIM / REFUND).
  • Receipts: 9 filings re-read from PayPal after filing on Nov 11; 9 matched what was approved (docs/sandbox-evidence/2026-11-11/receipts/).
  • Every call and its PayPal-Debug-Id is in docs/sandbox-evidence/2026-11-11/.

Challenges I ran into

The live sandbox found six bugs the fixtures never showed: an invoice id ending in a hyphen failed the verifier; lowercase invoice ids split by the verifier; provide-evidence rejected with ITEM_ID_IS_MANDATORY_FOR_MULTIPLE_EVIDENCES (now one merged evidence); the queue filter hid chargebacks under PayPal review; the CLI re-ran the LLM at approval, so the token never matched; and the LLM wrote “the address matches” and “the buyer received it” from a tracker that only said SHIPPED (two new verifier rules). Each one has a regression test built from the real sandbox response.

Accomplishments that I’m proud of

  • Offline evals on 24 synthetic disputes (labels frozen before the first run): always-refund 38%, always-fight 54%, Casefile 100% (in-sample). Requested evidence attached on FIGHT cases: 16/16 vs 0/16 for a generic reply. Verifier attack suite 12/12.
  • LLM notes accepted by the verifier on 13 FIGHT fixtures: 10/13 first draft, 12/13 after one repair round.
  • 9 of 9 live sandbox filings matched PayPal’s stored copy after the receipt check.

What I learned

A review-then-send step is not enough when an LLM can rewrite the notes at send time (7 of 8 live disputes drifted). Binding the approval token to the exact call and then reading PayPal’s own stored copy back is what made the receipt honest. The sandbox also taught us that fixture-only tests miss real API constraints (hyphenated invoice ids, ITEM_ID_IS_MANDATORY_FOR_MULTIPLE_EVIDENCES, links that appear a minute before PayPal will accept the filing).

Limits (honest; include in the description)

Other entries also draft evidence, verify claims and file with PayPal, and some keep hash-chained local audit logs; our addition is narrower: checking the filing against PayPal’s own stored copy. We could not check every entry (the gallery is not public yet). The same person wrote the labels and the rules, so 100% is a pipeline check, not proof the policy suits a real shop. The verifier checks numbers, ids and a few claim types; it cannot judge every qualitative sentence. Sandbox chargebacks could not be adjudicated (ACTION_NOT_ALLOWED_IN_CURRENT_DISPUTE_STATE), so no won/lost outcome is shown. No real seller has used Casefile yet.

What’s next

Webhooks instead of polling, make-offer for partial refunds, send-message to the buyer, a held-out case set written by someone else, and a test with a real small seller.

Built With

  • gpt-oss-120b
  • groq
  • html
  • llm
  • paypal-disputes-api
  • paypal-orders-v2
  • paypal-payments-v2
  • paypal-rest-api
  • paypal-sandbox
  • paypal-shipment-tracking
  • pdf
  • playwright
  • python
Share this project:

Updates

Submission history