Inspiration
Digital goods merchants face an asymmetric fraud problem: stolen accounts, card-testing rings, and friendly fraudsters pass checkout, download licenses, credits, or API keys instantly, and trigger chargebacks weeks later—costing the merchant the merchandise, the revenue, and the dispute fee.
We noticed that traditional checkout fraud filters run before payment authorization (causing false declines) or after capture and delivery (too late to stop fulfilment). With PayPal Orders v2 (intent=AUTHORIZE), there is a better moment to intervene: right after the buyer authorizes funds, and right before digital delivery unlocks.
FinShield turns that authorization hold into an intelligent, explainable pre-delivery security gate.
What It Does
FinShield intercepts PayPal Sandbox authorization holds before a merchant releases any digital product, evaluates the transaction across 5 independent fraud signals, and enforces one of four real-time fulfilment decisions:
$$ R(x) \in [0, 1] \implies \begin{cases} \text{Auto-Capture} & R(x) < 0.30 \quad (\text{ship immediately}) \ \text{OTP Step-Up} & 0.30 \le R(x) < 0.60 \quad (\text{verify buyer before capture}) \ \text{Analyst Review} & 0.60 \le R(x) < 0.85 \quad (\text{queue for human sign-off}) \ \text{Auto-Void} & R(x) \ge 0.85 \quad (\text{release PayPal hold before delivery}) \end{cases} $$
- Live PayPal Orders v2 Integration: Supports real PayPal Sandbox checkout (
intent=AUTHORIZE), immediateAuto-Capture($42clean buyer), step-up OTP gate ($220off-profile spend spike), human review queue, and zero-chargebackAuto-Void($4,800shared card-testing ring) plus PayPal refund settlement and dispute defense packets. - 5-Signal Risk Fusion Engine: Combines XGBoost (supervised), Isolation Forest (unsupervised anomaly detector), Behavioral Drift profiling, 8 deterministic policy rules, and a NetworkX Entity/Device Graph into one calibrated score.
- TreeSHAP Explainability & Forensic Graph: Every decision exposes per-feature SHAP contributions (
amount_deviation,velocity_5m, shared device IDs) and maps linked accounts sharing compromised hardware. - Guarded Human-in-the-Loop AI Analyst (
/agent): Analysts can ask natural-language questions, trigger simulated disputes, and inspect pre-built Audit Evidence Briefs (containing auth risk score, capture timing, and SHAP drivers). Safety guardrail: AI tools can only propose actions (propose_capture,propose_void,propose_refund)—PayPal money only moves when a human clicks Approve.
How We Built It
- ML & Risk Core (
Python,XGBoost,scikit-learn,SHAP,NetworkX): Trained and evaluated offline on the public ULB Credit Card Fraud dataset (284,807transactions,492fraud cases /0.17%positive rate) using zero-leakage stratified splits with scalers fitted strictly on the training split. - Backend API & PayPal Orchestration (
FastAPI,Pydantic,httpx,SQLite): Built REST endpoints for PayPal order creation, authorization risk gating, OTP challenge verification, webhook ingestion, forensic entity investigation, and the guarded analyst agent queue. - Workstation UI (
Next.js 16,React 19,TypeScript,Tailwind CSS): Built a responsive 5-surface console covering Overview (/), PayPal Sandbox Demo (/paypal), Analyst Agent (/agent), Operations Console (/command-center), Investigation Graph (/investigate), and Model Observatory (/performance).
Benchmarks & Accuracy (Offline ULB Validation)
| Model | ROC-AUC | PR-AUC | Precision | Recall | F1 Score |
|---|---|---|---|---|---|
| Logistic Regression (Baseline) | 0.9495 |
0.7005 |
0.8197 |
0.6757 |
0.7407 |
| XGBoost (500 Trees, Scaled) | 0.9709 |
0.8418 |
0.9206 |
0.7838 |
0.8467 |
Across 42,643 legitimate held-out orders, FinShield catches 58 true fraud cases with only 5 false positives.
Challenges We Ran Into
- Zero-Leakage Imbalanced Validation: Fraud sits at just
0.17%in real distributions. Fitting scalers before splitting or judging solely on ROC-AUC hides false alarms—so we enforced strict train-only scaling and tracked PR-AUC (0.8418) alongside synthetic overlap stress tests. - Preventing Prompt-Injected Money Movement: Letting an AI analyst directly call PayPal capture/refund APIs is unsafe in production. We structurally separated read-only investigation tools from money-moving execution at the router boundary so autonomous loops can never settle funds without explicit human approval.
- Timing Pre-Delivery PayPal States: Coordinating async PayPal Sandbox checkout redirects (
AUTHORIZE), instant local 5-signal scoring, OTP step-up holds, and ledger state transitions cleanly without leaving orphaned authorizations.
What We Learned
- Authorization holds are an underused sweet spot in digital commerce: voiding a
0.89risk hold before delivery eliminates the chargeback entirely instead of fighting a dispute weeks later. - Combining deterministic policy tripwires, graph clustering (
NetworkX), and calibrated ML (XGBoost+SHAP) gives analysts why a payment failed rather than an opaque black-box score.
What's Next
- Live merchant webhook plug-in for Shopify, Stripe, and PayPal Commerce Platform digital fulfilment triggers
- Automated dispute evidence PDF/API submission directly to PayPal Disputes API
- Real-time graph streaming for cross-merchant mule ring detection
Built With
- fastapi
- httpx
- machine-learning
- networkx
- nextjs
- pandas
- paypal
- paypal-orders-v2
- pydantic
- python
- react
- scikit-learn
- shap
- sqlite
- tailwindcss
- typescript
- xgboost