Preflight for PayPal Payouts

Every PayPal payout checked, explained, and stoppable before it leaves. AI reads and explains; a rules engine decides, so fake payment requests get stopped and genuine payees get paid.

Comment

Inspiration

Fake "please pay this new account" emails are business email compromise, which cost U.S. victims $3.05 billion in 2025 (FBI IC3, 2025 Internet Crime Report). They work for one simple reason: the business approves the payment itself. To PayPal it looks like a valid, authorised payout, so nothing downstream questions it.

Big companies buy tools for this. A small agency paying freelancers through PayPal has nothing between approve and send. I come from data engineering and data science, including fraud detection work, where fraud is usually found after the money has moved. I wanted to catch it before.

What it does

Preflight is a checkpoint between approve and send for businesses that pay many people through PayPal Payouts.

  1. Read the request. Paste an email, a list or a CSV, or upload the original .eml. AI turns it into a payout batch, and every amount and email must appear word for word in the source text, or that row is held.
  2. Check every row, without AI. A deterministic rules engine runs the owner's rules (written in plain English, read back and backtested on past payouts), compares each amount with that payee's own history, and checks the sender's headers: lookalike domain, reply-to redirect, SPF/DKIM/DMARC, pressure words.
  3. Decide. Each row is Clear (sent at once), Warning (sent after one click) or Held (never reaches PayPal). Only the engine sets this.
  4. Explain. For held rows, an AI investigator calls four tools and writes a verdict with cited evidence. It only advises.
  5. Resolve safely. A held payee gets a one-click check at the email on file, never the address in the request. If the payout details are new, an email "Yes" isn't enough: a code goes to the phone on file, because the mailbox itself may be what's hacked. Owners can release only after a cooling-off period and a call to the number on file.

It handles the real attack patterns, each one included as a one-click demo scenario:

Scenario What Preflight does
Fake sender (one letter off) Holds the whole batch; "Not from them: stop all" stops every payout in one click
Boss's mailbox hacked Sender checks pass, but the changed PayPal email is held and checked with the real payee
Payee's mailbox hacked The scammer's email "Yes" sends nothing; the code goes to the real payee's phone
Ghost contractor New payee over the limit is held, with nothing on file to confirm with
Inflated invoice / paid twice Held or warned against that payee's own history and the rest of the batch

Genuine payees aren't punished: only risky rows wait, warnings never block, baselines learn, and a payee verifies once.

More to try (beyond the three scenarios in the video):

  • Acknowledge a warning: Tom was paid the same amount 3 days ago, so his row needs one click before it's sent
  • Edit and re-check: fix an amount right in the AG Grid and the row is checked again instantly
  • Owner release: only after a cooling-off period and a call to the payee's number on file
  • Wrong codes: three wrong phone codes stop the payout as fraud
  • Verify once: a genuine payee who confirms new details goes straight through from then on
  • Plain-English rules: read back and backtested on past payouts before they go live
  • Sender check: upload the original .eml and its headers are checked in code
  • Audit log: every decision and action is recorded
  • Demo vs Product: Demo simulates the payee's email and phone; Product shows what the business actually sees

How I built it

  • PayPal Payouts API: POST /v1/payments/payouts for one batch per send (a unique sender_batch_id means a double click can't double-pay), GET for live item statuses (SUCCESS, UNCLAIMED, RETURNED) and the cancel endpoint for unclaimed items. Every payout carries the note "Checked by Preflight before it was sent", which shows up in the payee's PayPal email.
  • AG Studio + AG Grid for the whole interface: six custom widgets (scoreboard, review grid with sparklines, signal chips and live PayPal statuses, case file, sender check, rules and backtests, safety test), a Risk Control Room built from Studio's own charts over four related data sources, and a Preflight Analyst agent on the Agent Framework with five Preflight tools that hands dashboard work to Studio's built-in agents.
  • OpenAI Responses API with strict JSON schemas for extraction, rule compiling and the tool-using investigator, with code fallbacks when the AI is down or rate-limited.
  • Next.js and TypeScript on PayPal's official AG Studio boilerplate, hosted on Render.

Unusual amounts are scored against each payee's own history with a robust z-score:

$$ z = \frac{x - \tilde{x}}{1.4826 \cdot \mathrm{MAD}} $$

where x is the payout, x̃ is the payee's median payout and MAD is the median absolute deviation, with a floor of 5% of the median so a payee who is always paid the same amount isn't flagged for a cent. A warning needs z > 3.5 and an amount of at least 1.5× the payee's median.

A reproducible synthetic test (500 payouts, fixed seed) runs inside the app: 77% of genuine payouts went straight through, and 42 of 43 frauds were caught. The miss is reported too: an inflated amount to a real payee's real email that stayed inside their normal range.

Challenges

  • Keeping AI out of the money path. The AI reads, compiles rules and explains, but a deterministic engine sets every payout's state. An AI mistake can mislabel a case; it can't move money.
  • Not punishing genuine payees. Each extra check had to earn its place: holds only where signals stack, a duplicate rule that knows weekly payees are paid weekly, and a 14-day warm-up so backtests don't flag the start of history.
  • Thinking like an attacker. My first version verified held payees by email, until I asked "what if the payee's mailbox is the thing that's hacked?" That led to the code on the phone on file. "What if the sender is fake?" led to holding the whole batch.
  • Sandbox realities. Venmo isn't supported as a wallet in the sandbox, cancel returned internal errors, and sandbox emails are shared across developers. Preflight labels and handles each one honestly instead of hiding it.
  • AI rate limits. I hit the daily request cap during testing, so extraction and rules fall back to code, and only held rows are investigated.

What I learned

Payment safety with AI is mostly about where AI is allowed to act. The interesting design work wasn't the model; it was the boundary: what the AI can read, what it can suggest, and the one thing it can never do.

What's next

Payouts webhooks instead of polling, a database for state, real email and SMS delivery for verification, and Preflight in front of PayPal's MCP server, so AI agents can only send money through it.

Built With

Share this project:

Updates

Submission history