Inspiration
Small online merchants spend dozens of hours every month wrestling with fragmented dashboards, exporting messy CSVs, and missing subtle fraud patterns until costly chargebacks hit. When customers file payment disputes, merchants often lose by default simply because assembling order evidence, tracking receipts, and cross-referencing their own return policies is slow and overwhelming.
We asked: What if every small online merchant had an autonomous AI financial officer living inside their PayPal ledger?
We built PayPal Merchant Copilot to give sellers three superpowers: the ability to query their financial data in plain English, proactive machine-learning anomaly detection to stop fraud before chargebacks occur, and an AI dispute defense assistant grounded in the merchant’s own store policies.
What it does
PayPal Merchant Copilot runs directly on top of the PayPal Sandbox platform and delivers three core operational capabilities:
- Ask Your Money (Guarded NL-2-SQL): Merchants ask plain-English questions ("Which item had the most refunds this month?", "Revenue by buyer country", "Top selling products"). The copilot translates the question into guarded, read-only SQL, executes it against the ledger, and renders dynamic bar charts, trendlines, and tables with full explainability.
- Catch Risk Early (Explainable Anomaly Detection): A hybrid risk engine combining heuristic rules and an Isolation Forest ML model continuously scans transactions for:
- Payment velocity bursts (e.g., 8 rapid micro-payments within minutes).
- Duplicate charges (identical amounts within 10 minutes).
- Unusual order amounts (Z-score outlier detection per item).
- High-value orders from rare geographic regions.
- Sudden refund spikes against historical 7-day baselines. Every flag provides a clear, plain-English reason so merchants can confirm or dismiss risks with one click.
- Dispute Defense Assistant (Policy RAG): When a customer opens a dispute, the copilot extracts order history, checks for duplicates or prior refunds, and uses TF-IDF cosine similarity to retrieve exact sections from the merchant’s refund and shipping policies (
data/policies/*.md). It then drafts a factual, policy-cited response ready to submit to the PayPal Resolution Center. - Dual-Mode Engine: Operates in Live Sandbox Mode (live Orders v2 checkout, Reporting API sync, and PayPal webhook signature verification) or Zero-Friction Mock Mode (generating authentic webhook payloads for instant offline judging demonstrations).
How we built it
- Backend Architecture: Built with FastAPI (Python 3.11) with an asynchronous design and clean modular services (
paypal.py,ledger.py,nl2sql.py,anomaly.py,disputes.py,seed.py). - PayPal Platform Integration: Implemented PayPal REST APIs including Orders v2 (create order, customer approval redirect, server-side capture), Reporting API (
/v1/reporting/transactionssync), Disputes API, and cryptographic Webhook Signature Verification (/v1/notifications/verify-webhook-signature). - Data & Idempotency Layer: SQLite ledger with strict event-level idempotency (
webhook_events,paypal_txn_id), ensuring zero double-counting even under duplicate webhook retries. - Multi-Layer NL-2-SQL Security Guard: Built an AST parse-tree guard using sqlglot that enforces: single statement only, strictly
SELECT, strict table allowlisting, zeroWITHmutations, and execution on a dedicated read-only SQLite connection (PRAGMA query_only = ON) with execution timeouts and row limits. - Machine Learning & RAG Pipeline:
- Unsupervised outlier scoring via scikit-learn Isolation Forest combined with Z-score feature engineering.
- TF-IDF vectorization and cosine similarity over markdown policy chunks for factual retrieval.
- Anthropic Claude 3.5 Sonnet integration with an intelligent fallback to deterministic templates.
- Executive UI: Responsive, zero-dependency fintech workstation with dark/light theme switching, live ledger tape with real-time text filtering, dynamic SVG sparklines, and policy review modals.
Challenges we ran into
- Guaranteed SQL Safety & Anti-Hallucination: LLMs can easily hallucinate destructive SQL (
DROP,DELETE, or leaky subqueries). We solved this with a defense-in-depth approach: AST parsing withsqlglotrejects invalid statements before execution, and the database connection is physically read-only at the OS and SQLite pragma levels. - Webhook Idempotency & Cloud Networking: PayPal webhooks are asynchronous and can arrive out of order or retry multiple times. We built an idempotent event consumer that deduplicates incoming events by
event_idand integrated a Cloudflare Tunnel for secure local receipt of PayPal webhook events. - Eliminating Hallucinations in Dispute Defense: A legal response must never fabricate tracking numbers or dates. We constrained prompt generation strictly to retrieved ledger facts (timestamps, duplicate matching, prior order count) and cited policy passages, backed by a deterministic fallback that ensures every dispute gets a cited defense even with zero LLM credits.
Accomplishments that we're proud of
- 17 Automated Integration Tests Passing (100% Pass Rate): Comprehensive pytest test suite validating that SQL injection attacks are rejected, read-only connections block writes, revenue totals match direct SQLite queries, and dispute drafts cite legitimate policies.
- High-Precision Risk Benchmark: Evaluated on a deterministic 60-day ledger (971 transactions, 4 planted fraud patterns): the anomaly engine detected all 4 planted fraud types with 100% recall and zero false positives.
- Instant Demoability: The entire app can be spun up in 2 minutes with zero API keys in Mock Mode, or plugged into live PayPal Sandbox credentials with a single
.envupdate.
What we learned
- How to architect airtight, read-only NL-2-SQL engines where untrusted model outputs can safely interact with financial ledgers.
- The deep mechanics of PayPal’s Orders v2 lifecycle, webhook cryptographic signature verification, and cross-border payment compliance.
- That combining explainable heuristic business rules with unsupervised ML (Isolation Forest) delivers far superior real-world usability than black-box models alone: rules provide instant human interpretability, while ML catches subtle multivariate outliers.
What's next for PayPal Merchant Copilot
- Direct Two-Way Dispute Submission: Wire the "Approve Draft" button directly to PayPal’s Disputes API to submit evidence packets into the PayPal Resolution Center with zero manual copy-pasting.
- Predictive Cashflow Forecasting: Extend the ledger engine with time-series forecasting (ARIMA / Prophet) to predict 30-day cash reserves, upcoming subscription rebills, and anticipated refund liability.
- Automated Cross-Border Tax & VAT Advisory: Auto-categorize cross-border transactions and prepare export tax summaries for international PayPal sellers.
Built With
- anthropic-claude
- cloudflare-tunnel
- css3
- docker
- fastapi
- html5
- httpx
- javascript
- numpy
- pandas
- paypal
- pytest
- python
- scikit-learn
- sqlglot
- sqlite
- uvicorn