Inspiration
A fake invoice is dangerous because it is a real PayPal document. A stranger sends an invoice that says "your antivirus subscription was renewed, call this number to cancel", or a normal invoice is quietly edited after it was sent. The usual "is this page fake?" checks find nothing, because the page is genuine. We wanted a tool that looks at the invoice itself, compares it with what it already knows about the sender, and tells the person exactly why it looks wrong, so they decide instead of trusting a number.
What it does
Scam Copilot is a Chrome extension (Manifest V3) with a FastAPI backend. It looks at one event from both sides:
- As the payer. On the invoice page PayPal shows you, press Check this PayPal invoice. The extension reads the visible text of that page, only when you press the button, and the backend turns it into invoice fields. On a PayPal invoice email, press Check this email: it checks where the links lead, what changed since the earlier email about the same invoice, and compares the page with the emails. That catches, for example, a note changed after sending, which PayPal's update email leaves out.
- As the issuer. The backend watches the connected account's invoices through the PayPal Invoicing API (and optional signed webhooks), stores every version of each invoice, and checks each new or changed one. A new risky invoice shows up on its own, with an alert.
- Around the checks. Every check is kept in a History table, in-app alerts follow rules you set, the Monitor tab shows live state, and a monthly report lists what is worth a manual look.
- Paying, with checks first. In History, an invoice with no warning signs gets a Pay button and a checkbox; invoices with warning signs cannot be ticked. "Pay N invoices" opens PayPal's own payment page for each chosen invoice, after the server has read every invoice again and refused any that changed since its check. Scam Copilot never moves money: PayPal asks for the payment on its page, and an invoice counts as paid only when PayPal says so. You can also ask the chat to "pay all the low risk invoices": the assistant only prepares the list and sends you to History.
Every finding carries its evidence. The wording is always "signs found", never "fraud", and a field the page or API did not show stays empty instead of being guessed.
Where AI comes in. The rules decide the risk level; the model never changes the score. A language model explains the findings in plain language and powers a chat. It sees only minimised data, the invoice text reaches it as data and never as instructions, and anything in its answer that is not in the input (an email, link, phone number or amount) is removed. An answer that states fraud or safety as a fact is replaced by what the rules found. In the chat the model can also call two read-only tools of the PayPal Agent Toolkit, list_invoices and get_invoice (Sandbox only), so questions like "which invoices are waiting for payment?" are answered from live account data.
How we built it
The side panel is React and TypeScript. The History, alerts, live events and monthly report tables run on AG Grid Community, with filter, sort, quick search and CSV export, and new rows appear in the Monitor tab as soon as the backend sees a new invoice. The backend is FastAPI, SQLAlchemy and SQLite, with the PayPal Invoicing API v2 and webhooks in the Sandbox.
Scoring is deterministic: a rule engine for invoices and a separate set of rules for emails (links, changes between versions, page versus email). PayPal is stubbed in tests, so 391 backend tests run without a network. A 59-case evaluation set, written by us, covers normal, suspicious, duplicate and insufficient-data invoices.
The AI layer uses free OpenRouter models only: the provider refuses any model whose id does not end in :free or whose price is not zero, and tries the next model in a list when one is rate-limited. The Toolkit runs as @paypal/mcp, pinned by a lockfile and started with only the two read tools. Results are masked and trimmed before the model sees them.
The project was created during the submission period (first commit 18 October 2026) by a team of four.
Challenges we ran into
- The API shows only the issuer's side. The Invoicing API returns invoices issued by the connected account; an invoice from someone else answers
403, and Log in with PayPal does not grant the invoicing scope. So the payer's side reads the page and the email, and we say so. - Webhooks are a hint, not data. In our Sandbox tests events reached only the issuer's app, arrived in batches and out of order, and carried the invoice as of delivery time. We treat an event as a signal, read the invoice again, and store each version once by content hash.
- Scoring depends on history. An invoice is compared with earlier ones from the same supplier, so order matters. The bulk check processes invoices oldest first so a re-run gives the same levels.
- Emails. PayPal's update email omits the seller's note, and Sandbox emails are shown only in the Developer Dashboard, so the email path was built and tested on saved and hand-opened emails.
- A model reading a stranger's text. Free models are rate-limited at any time and sometimes invent a finding, which is why the rules own the score and every model claim is checked against the input.
- Tool calls on free models. Only some free models return tool calls reliably, so after
list_invoicesthe next step is forced and the invoice table is built by code from the full data, while the model writes one or two sentences.
Accomplishments that we're proud of
- One event, seen from both sides. The issuer's side comes from the Invoicing API and signed webhooks, the payer's side from the page and the email, and the two are compared. We built it around what PayPal's platform really exposes, not around what we wished it did.
- Every sign has its evidence. No black-box score: each finding says what it found and where, and missing data is shown as missing. The model explains, the rules decide.
- A real webhook path. We verified PayPal's signature against live Sandbox events (a genuine event passed, one with a changed amount failed), and we handle events that arrive in batches and out of order without storing a version twice.
- Safe use of a model on strangers' text. Prompt-injection tests, answers checked against the input, a provider that refuses any model that is not provably free, and a guard that replaces unsupported claims of fraud or safety.
- A chat grounded in live data. It answers questions about the account's own invoices through the PayPal Agent Toolkit, read-only and Sandbox only, and shows which tool it used.
- Tables that behave like tables. Sorting, filtering, CSV export and live rows on AG Grid, in a narrow side panel.
- A tested, reproducible build. 391 backend tests that need no network, and a clean install from the README.
What we learned
Evidence beats confidence: a short list of readable reasons is more useful than a score. Treat everything from a stranger as data, including inside a prompt. Treat a webhook as a hint and read the source of truth. And be upfront about what the platform can and cannot see; it made the product more honest and the demo easier to trust.
What's next for scam-copilot
First, the limits we know about: it runs on the PayPal Sandbox only and has never been tested against live accounts; the payer path reads what the page and the email show (the page has no issuer phone and often no issuer email); the rules are hand-tuned on demo scenarios, not on real fraud data, and we have not measured their precision on real invoices; and a low score does not prove an invoice is safe.
Next: let a payer connect their own account if PayPal opens that access, tune the rules on real data, add vendor profiles and duplicate detection across accounts, run a locally hosted model so no invoice leaves the machine, and offer a hosted deployment.
Built With
- ag-grid
- chrome
- fastapi
- node.js
- openrouter
- paypal-agent-toolkit
- paypal-sandbox
- paypal-webhooks
- pydantic
- pytest
- python
- react
- sqlalchemy
- sqlite
- tailwindcss
- typescript
- uvicorn
- vite
