ProofPilot

Inspiration

Small sellers often lose PayPal disputes on paperwork, not merits. PayPal's own help pages say new sellers' payments can be held "for up to 21 days" and that adding tracking releases a hold "approximately 24 hours after the courier confirms delivery" (sources: https://www.paypal.com/us/cshelp/article/HELP848 and https://www.paypal.com/us/cshelp/article/help129). Seller Protection depends on details sellers rarely check before shipping, and dispute responses depend on supplying exactly the evidence PayPal requests before a deadline. We wanted a tool that helps sellers work with PayPal's process correctly, before and after a sale.

What it does

• Before shipping: re-reads each order and capture from PayPal and flags things that can void Seller Protection (address redirect, missing shipping address, capture not completed, not eligible, USD 750+ without signature, digital goods). Then it pushes tracking to PayPal.
• When a dispute arrives: shows it in a queue with PayPal's real deadline and the evidence PayPal requests, in plain words.
• Evidence pack: every statement is generated from a real PayPal record (order, capture, tracker, refund) and cites it. A checker refuses anything without a matching record, any amount, date or id not in the cited record, outcome predictions, and tracking added after the dispute opened. Buyer-typed text is shown only as a labelled quote.
• AI agent: built on the PayPal Agent Toolkit with Gemini. It reads the case with read tools on its own and recommends fight, offer, accept or wait. It cannot write evidence text, and every money-moving write (refund, accept, offer) becomes a proposal bound to that dispute's own payment, capped per payment with pending refunds counted, and runs only with a single-use approval code. Evidence is sent only after the seller reviews the pack and types APPROVE.
• Seller approval: nothing is sent to PayPal until the seller approves the exact pack. After submitting, the case shows "Under PayPal review". ProofPilot never claims an outcome.

How we built it

Node 20 server, PayPal sandbox REST APIs, AG Grid Community for the order and dispute grids, Vercel AI SDK 5 with Gemini driving the PayPal Agent Toolkit, and a deterministic rules engine as fallback and sanity check.

Tools used (required: which tools and how)

Challenges we ran into

• Gemini 3.x tool calling needed thought signatures passed between steps, so we moved to AI SDK 5.
• Prompt injection through buyer-typed fields: an address line could read like a seller confession. We now treat buyer text as untrusted quoted data.
• Keeping the AI honest: the model picks fact IDs and ProofPilot writes the words, after an independent reviewer showed free-text checking could be fooled.
• In the sandbox, PayPal refused one evidence type at the claim stage. ProofPilot now remembers refusals and stops recommending them.

Accomplishments

An in-app sandbox offer (run on a local instance), recommended by the rules engine and approved with a code a person typed, was accepted by PayPal: the case moved from waiting for the seller to waiting for the buyer. A team bot then played the sandbox buyer (not a real customer) and accepted the $9 offer, and PayPal marked dispute PP-R-MVK-10191109 resolved with a $9 partial refund. In-app evidence submissions were refused by PayPal at the chargeback stage, and ProofPilot recorded the refusals and stopped recommending that type. Hosted on Render from commit db36236. On the hosted app, Gemini ran live through the PayPal Agent Toolkit on a sandbox dispute: it read the dispute and produced a proposal that waits for a person's one-time code, and nothing was written to PayPal. When Gemini was rate-limited during the demo recording, the Groq backup (openai/gpt-oss-20b) answered live as an advisor with no write tools. Webhook delivery is verified: PayPal's PAYMENT.CAPTURE.REFUNDED and CUSTOMER.DISPUTE.RESOLVED events reached the hosted app with signature verification SUCCESS, and both were still there after a restart with Postgres history enabled. 133 automated tests (132 pass, 1 skipped, at commit db36236), adversarial test suites from an independent review. Gitleaks finds no secrets across all 22 commits of the published history.

What we learned

Timing and provenance of evidence matter as much as the evidence. Agents that touch money need hard limits in code, not just in the prompt.

What's next

Live-mode onboarding with PayPal's approval, more Seller Protection checks, and support for more dispute reasons.

Testing instructions

• Hosted demo: https://proofpilot-cbgr.onrender.com (free plan, first load after idle takes about a minute) with reviewer passcode [given in Devpost private field].
• Local: npm install, set PAYPAL_SANDBOX_CLIENT_ID and PAYPAL_SANDBOX_CLIENT_SECRET from your own sandbox app, npm start.
• A sandbox buyer login (test money only) for paying in the demo shop is in the Devpost private field, next to the reviewer passcode. You can also use a personal account from your own sandbox.
• Limits (disclosed): sandbox only; on the hosted demo only the host owner can approve agent writes, because approval codes appear only in the server console; Gemini's free tier has a small daily quota, so answers are labelled LIVE, RULES ENGINE FALLBACK, or CACHED DEMO RUN (recorded earlier, not live).
• The hosted AI runs on a free-tier model quota; when the quota is hit, ProofPilot falls back to its rules engine, shows a red "Rules engine fallback" banner with the reason and a Try again button, and writes nothing to PayPal.
• Saved history needs the optional DATABASE_URL Postgres; without it the hosted app falls back to a local file that resets on restart.
• The shop's order-result endpoint (GET /api/orders/:id/result) has no passcode or rate limit; it rejects any id that isn't a 17-character PayPal-style id, returns only a summary (order and capture ids, status, risk check), never payer email or address, and relies on PayPal order ids being hard to guess.
• Not legal advice.

Built With

• ag-grid
• gemini
• groq
• javascript
• node.js
• paypal
• postgresql
• render
• vercel-ai-sdk