Inspiration
In October I took part in a 3-day AI hackathon with CGI on anti-money-laundering, and saw how hard it is to detect fraud and explain it. Small PayPal sellers face this every day, without a fraud team. When a fraudster pays with a stolen card, the seller loses both the money and the product. I wanted to give them an AI assistant they can trust.
What it does
What makes PayGuard different: a real machine learning model (not just an LLM opinion) with exact SHAP reasons for every alert, a fairness rule (the model never judges nationality), and the full seller journey on PayPal: payment, fraud review, refund and dispute. The LLM never decides money: refund proposals come from fixed thresholds, and only the seller can confirm them.
PayGuard AI watches a seller's PayPal payments and:
- gives each payment a risk score from 0 to 100 and explains it in plain English;
- lets the seller talk to Ask PayGuard, an AI agent that chooses its own tools (including the official PayPal Agent Toolkit), reviews the payments and prepares refunds;
- helps answer PayPal disputes with an evidence checklist and a draft reply;
- never moves money or sends anything without the seller's explicit confirmation.
How we built it
- PayPal (sandbox): Orders v2 (create and capture), Payments v2 (refunds), webhooks with signature verification, a demo shop with the PayPal JavaScript SDK, the Disputes API, and the PayPal Agent Toolkit.
- Risk model: LightGBM trained on the public PaySim dataset with a temporal split, using only signals a seller can compute (amount vs. usual ticket, payer history, cross-border or domestic).
- Explainability: exact TreeSHAP reasons for every alert.
- LLM: Groq (
gpt-oss-120b) writes the explanations, runs the agent with function calling, and drafts dispute replies. Every number is checked against the tools' results; if a check fails, a safe template is used. - App: Streamlit, Docker, deployed on Render, with 160+ automated tests.
Challenges we ran into
- The first model relied on the hour of day, a simulator artifact: at night every payment looked risky. I removed it so results are the same at any time.
- An early version scored countries differently. I replaced it with a simple cross-border signal, so the model never judges nationality.
- The LLM sometimes invented numbers or claimed it had sent things it had not. I added strict checks, retries and fallbacks.
Accomplishments that we're proud of
A working end-to-end flow on real PayPal sandbox payments, refunds and webhooks, an AI agent with real guardrails, and an honest README.
What we learned
Choosing the right metric matters as much as choosing the model, and an AI agent that touches money must keep a human in control.
What's next for PayGuard AI
Train on real seller data, add business rules next to the model, and send evidence files for disputes.
