pre-dispute

Pre-Dispute is an AI agent that fixes late orders and worried buyers before they become PayPal disputes, within merchant-set limits. You pay only for disputes it prevents.

Comment

Inspiration

When an online order goes wrong, a late package, a silent seller, a worried buyer, small merchants usually find out when the PayPal dispute or chargeback lands. By then the cheap fixes are gone: one honest message or a small goodwill refund would have saved the sale, but now there are fees, evidence to gather and an account at risk. Fraud tooling scores transactions well, but little of it watches the quiet stretch between payment captured and dispute opened. That gap is where Pre-Dispute lives.

What it does

When an online order goes wrong, a late package, a silent seller, a worried buyer, small merchants usually find out when the PayPal dispute or chargeback lands. By then the cheap fixes are gone: one honest message or a small goodwill refund would have saved the sale, but now there are fees, evidence to gather and an account at risk. Fraud tooling scores transactions well, but little of it watches the quiet stretch between payment captured and dispute opened. That gap is where Pre-Dispute lives.

What it does

Pre-Dispute watches paid PayPal orders. When a buyer signal arrives, a Gemini agent reads the case and proposes the cheapest fix: a status update, a partial refund, a reship or a full refund.

The AI only proposes. A deterministic policy engine decides what may run unattended:

  • refunds up to €10 and up to 30% of the order, with risk below 0.8, run automatically through PayPal's Refunds API;
  • anything larger, any reship, and any high-risk case (risk 0.8 or more, such as a dispute threat) goes to a merchant approval queue;
  • the agent is never told the actual limits, so it can't game them.

Billing is outcome-based. A case the agent judged at risk of 50% or more counts as a prevented dispute after 14 quiet days (in the demo, a button fast-forwards this), and prevented cases are netted into one PayPal Invoice. If a dispute does arrive, a signature-verified PayPal webhook voids that order from billing, as long as it hasn't been invoiced yet.

The savings figure compares each outcome with the worst case, losing the dispute. The €15 is an assumed dispute handling cost:

$$\text{saved} = (\text{order} + 15) - \text{refund paid}$$

So a full refund shows little saving and a small goodwill refund shows a lot, and the dashboard labels the result an estimate.

How we built it

  • PayPal (sandbox): Orders v2 (demo orders are paid with the sandbox test card through payment_source.card, so they are real sandbox payments, captured), Payments v2 refunds with idempotency keys, Invoicing v2, and Webhooks with verify-webhook-signature on every event.
  • AI: Google Gemini, returning JSON validated against a schema. It tries several models in order and falls back to a clearly labelled rules-based fallback if every model is unavailable.
  • App: Node 24, Express, TypeScript, built-in node:sqlite.
  • Dashboard: AG Grid with custom badge and risk-meter renderers, and a case panel that shows the buyer's message, the AI's reasoning and each guardrail check.
  • Hosting: Render.

Challenges we ran into

  • The agent gamed the limits. Told that refunds above €10 need approval, it picked exactly €10 for an angry buyer on a €120 order. We stopped telling the model the limits, moved all authority into the policy engine, and added a hard rule that high-risk cases always reach the merchant.
  • It invented facts. One reply said "We've checked with the carrier, and your headphones are expected to arrive within 2 to 3 business days", though it had no carrier data. The prompt now says the agent only knows what it is given.
  • Idempotency keys outlived our database. Keys based on a counter that restarted collided with PayPal's memory of old keys, so the first refund after a reset was rejected as a duplicate. Keys are now tied to the capture ID.
  • Free-tier AI quotas can be as low as 20 requests per model per day, and models sometimes stall. We added timeouts, per-model cooldowns and the labelled fallback.

Accomplishments that we're proud of

  • Real PayPal API flows, end to end, in the sandbox. A captured order becomes an AI-proposed refund, then a policy check, then a merchant approval, then a PayPal refund, and finally one PayPal invoice for the outcomes.
  • An AI that can't exceed the merchant's rules. The model only proposes. A deterministic policy engine decides what runs unattended, and every decision shows its reasoning and each guardrail check on screen.
  • Signature-verified webhooks. The app registers its own webhook at boot, real signed PayPal events pass verification on the live site, and forged ones are rejected.
  • Honest numbers. The dashboard compares each outcome with the worst case of losing the dispute, so a full refund shows little saving and a small goodwill refund shows a lot. We label the figure as an estimate.
  • A demo that keeps working when a model is out of quota, thanks to timeouts, multi-model fallback and a clearly labelled rules-based fallback.
  • A live, public, open-source build that anyone can open and try in a minute, with no login.

What we learned

  • An agent that moves money needs what banks have: a separation between the party that proposes and the party that authorizes, with the reasoning visible. Giving the model less authority made the product more credible, not less useful.
  • In our tests the model gamed limits it could see and invented plausible facts, so the safest design is one where it never sees the limits and the system never trusts its claims.
  • Idempotency keys, timeouts and quota handling decide whether a demo survives contact with real users, and they're easy to overlook until something breaks.
  • Being upfront about what's simulated (shipping events, the 14-day check) makes the parts that are real easier to trust.

What's next for pre-dispute

  • Confirm "prevented" with the PayPal Disputes API and real carrier events, instead of a fast-forward button.
  • Let merchants configure their own limits and tone, per product or customer segment.
  • Support multiple merchants, each with its own policy and invoice.
  • Pilot pay-per-outcome pricing with real small merchants and measure true dispute and chargeback rates before and after.

Built With

Share this project:

Updates

Submission history