Exit code 0

PayPal checks the ID. PayoutGuard checks the person.

Comment

Inspiration

Payout runs for sellers, creators and contractors are CSVs, and mistakes are found after the money has moved. We tested what PayPal's sandbox does with them. A mistyped email is accepted and ends UNCLAIMED, with funds returned only after 30 days. The same sender_item_id twice gets its second item FAILED (DUPLICATE_ITEM). The same receiver under two different item IDs was paid twice, both SUCCESS, with no warning. PayPal's duplicate check works on IDs. Nothing checks whether two records are the same person.

What it does

PayoutGuard scores a payout batch against payout history before anything is sent.

  • Flags: duplicate payee (names and emails that differ but refer to the same person), typo receiver (an email one character off a known payee), unusual amount for that payee, and invalid input.
  • Evidence: every flag has a calibrated probability and a written explanation built from the evidence.
  • Approval: the grid opens with flagged rows first. The operator approves or holds each one. Send is disabled while a flagged row is undecided, and the server enforces the same rule.
  • Confirm step: it shows the item count, the total, and every row that will not be sent.
  • Payout: only approved rows reach the PayPal Payouts API. Each item's real PayPal status is shown, and an UNCLAIMED or FAILED item is never displayed as paid.
  • Audit log: every upload, score, decision, payout request and status change is recorded in an append-only log.

How we built it

  • ML (Python, scikit-learn, rapidfuzz):
    • Candidate pairs from blocking, then name-, email- and context-based pairwise features.
    • A gradient-boosted matcher with probabilities calibrated by cross-validated Brier score.
    • A per-payee amount model: a robust log-space z-score with empirical-Bayes shrinkage, and a cohort fallback for payees with thin history.
    • Deterministic rules for invalid input and exact duplicates.
    • No LLM is in the decision path.
  • Backend (FastAPI, SQLite):
    • A server-side approval gate: only approved rows are sent.
    • A deterministic sender_batch_id and idempotent retries.
    • It refuses any non-sandbox PayPal URL.
    • USD only.
  • PayPal: the Payouts API (POST /v1/payments/payouts and status polling), OAuth 2.0 client credentials, and sandbox Business and Personal accounts.
  • Frontend (React, Vite): AG Grid Community for the review, payout and audit tables, and Recharts for the evaluation graphs.
  • Data: a synthetic ledger generator, so no real payee data is used.

Live sandbox run

A 9-row slice of a 212-row demo batch, with 599 history rows. The model gave 3 proceed, 2 review and 4 hold. The operator approved 3 rows (37.48 USD) and held 6 (585.62 USD). PayPal returned SUCCESS for all 3 sent items.

Challenges

  • Payouts is not available from every country. Our first batch failed with 403 PAYOUT_NOT_AVAILABLE. PayPal's documentation lists India as "Receive and withdraw" only, so a sandbox Business account created in India cannot send payouts. We rebuilt the sandbox with a US Business account and a funded balance.
  • A train/serve skew bug. A country feature behaved differently at serving time, and aliases and typos were missed when scored through the batch interface (0 of 49 and 0 of 21). We removed the feature and retrained; the contract tests then caught 49 of 49 and 21 of 21. Our first regression test did not catch the bug, so we replaced it with an invariance test.
  • The sandbox showed a real bug in our retry path. On a reused sender_batch_id, PayPal returns 400 with the link to the original batch under details[].link[], not in top-level links.
  • Smaller ones: Windows CRLF line endings changed our ledger hash, and an OAuth invalid_client error took time to track down.

What we learned

PayPal's checks are ID-based, so "same person, different record" has to be checked by you. A batch can report SUCCESS while an item is FAILED or UNCLAIMED, so item status is what matters. On our synthetic evaluation, a logistic regression on the same features scored average precision 0.987 against 0.993 for the gradient-boosted model: the gain is small, and we say so on the Evaluation screen.

Limits

  • The evaluation uses synthetic data, so the numbers show the method works, not what it would score on your payouts.
  • The PayPal observations come from one sandbox run in USD.
  • It is sandbox only, and sends only USD.
  • The amount threshold (z = 8) was chosen by best F1 on the synthetic set. Lower values flag more rows (9% of the demo batch at z = 4).

What's next

Evaluation on real payout history with the owner's consent, non-USD payouts, and a hosted demo with its own isolated sandbox account.

Built With

Share this project:

Updates

Submission history