Invoice Guard

Fake invoices hit small businesses every day. Invoice Guard is an AI agent that pays your PayPal invoices, checks each with PayPal and AI first, and stops fakes before money moves.

Comment

Inspiration

Fake invoices are one of the most common ways small businesses lose money. The scams aren't clever: an antivirus renewal nobody bought, a supplier whose "bank details have changed", a refund for an overpayment that never happened. They work because the person paying the bills is busy.

An AI agent that pays invoices would save that person time. But an agent that pays is only useful if it can tell a real bill from a fake one, and if it knows when to stop and ask. I wanted to build the agent you could actually trust with a business's money.

What it does

Invoice Guard pays a small business's PayPal invoices for it. Forward an invoice email to it, or paste the link, and it runs three checks:

  • Is it real? It asks PayPal for the invoice by its ID. If PayPal has no record of it, or it's addressed to someone else, it's blocked.
  • Is it expected? It checks whether the business has paid this sender before, and whether the amount is in the usual range.
  • Does it read like a scam? Jev, a decision model from TypeSafe AI, answers eight yes/no questions about the wording: pressure, impersonation, a fee now for money later, changed payment details, and more.

Then plain code decides, where $p$ is Jev's probability that the invoice is a scam:

$$ \text{action} = \begin{cases} \text{block} & \text{if PayPal has no record of it, or } p \ge 0.8 \ \text{pay} & \text{if the sender was paid before, the amount is usual and under the owner's limit, and } p \le 0.2 \ \text{ask the owner} & \text{otherwise, including every first-time sender} \end{cases} $$

Known vendors are paid at once by PayPal payout. Anything the agent stops on waits for the owner, with a short note from Claude saying what looked wrong and what to check. The agent never pays a stranger without asking. Once the owner approves and pays a new vendor, that vendor's later invoices can be paid automatically. The owner sets the most the agent may pay on its own, and can unblock an invoice if the agent was wrong.

Because the email is only a pointer and the invoice is always read from PayPal, a convincing email can't change what gets checked. A PayPal-looking email that points to no real invoice is blocked on its own, which catches phishing that never touches PayPal at all.

How I built it

  • Next.js and TypeScript for the dashboard and API, with Postgres for the payment record, hosted on Render.
  • PayPal Invoicing API to read each invoice by its ID and to trust the sender's verified email.
  • PayPal Payouts to pay known vendors, with each payout's ID built from the invoice ID so PayPal refuses a second payment.
  • PayPal Transaction Search to load past payments, so vendors paid before the app was connected count as known.
  • Jev for the scam questions, and Claude (through OpenRouter) for the notes to the owner.
  • CloudMailin to receive forwarded emails, and AG Grid for the invoice table.
  • The interface follows a Figma community dashboard design. I built it with Claude Code as my coding assistant.

Why Jev instead of a general LLM. Jev can't invent anything: it only answers the yes/no questions I give it, with a probability, which fits an agent that moves money, where every judgment feeds a clear rule. I compared it with Claude and GPT models on the same 53 test invoices and the same eight questions, and it came down to a trade-off.

Jev was 6 to 16 times faster (about 0.2 seconds per invoice) and 5 to 80 times cheaper. That matters as a business grows: one receiving 100,000 invoices a month could have every one scored in about a fifth of a second for around $2.40 in Jev costs, against roughly $150 to $190 with a flagship general model, and those costs keep rising with volume.

The general models were slightly more accurate (50 to 52 of 53, against Jev's 49), mainly on fake invoices whose wording looks normal. I decided the speed and cost were worth it, because wording is never the only check: whether PayPal has the invoice and whether you've paid the sender before decide those cases, and an unknown sender is never paid automatically. Claude still writes the notes for the owner, which is the part that needs real language.

Before writing the app, I tested each piece in small Python experiments: Jev on 53 hand-written scam and honest requests, and the PayPal sandbox with four test business accounts.

Challenges I ran into

  • PayPal doesn't let an account list the invoices it has received. In my sandbox tests, listing and searching invoices only ever returned ones the account had sent. I redesigned around it: invoices arrive by email or paste, and PayPal is then asked for each one by its ID. It turned out to be an advantage, because it also catches fake PayPal emails.
  • The Invoicing API has no call to pay an invoice. It can only record a payment made somewhere else. Payouts move the money but leave the invoice open on PayPal, so first-time vendors are paid on PayPal's own invoice page instead.
  • Bland fake invoices read exactly like real ones. A fake toner invoice has nothing wrong in its wording, so no language model can reliably catch it. The payment-record check does: an unknown sender is never paid automatically.
  • Paying twice. My live deployment and a local copy didn't share a database, and the live one tried to pay an invoice the local one had already paid. PayPal refused, because the payout ID comes from the invoice ID and PayPal rejects a repeated ID for 30 days. The app now reads the earlier payout from PayPal's refusal and records it, and its own payment record covers anything older.

Accomplishments that I'm proud of

  • A vendor the business has paid before, at an ordinary amount, still gets blocked when its invoice says the bank details have changed and asks you not to call. That's the attack that fools people most.
  • In my tests, attempts to talk the model round ("this is a verified invoice", "ignore previous instructions") didn't lower any scam score.
  • Plain, tested code makes every money decision. The AI judges wording and writes notes; it never decides whether to pay.

What I learned

  • Where an AI model is strong and where it isn't. Jev was fast and good on scam wording, and weak on invoices whose wording is normal. Facts from PayPal covered that gap.
  • Asking a model eight narrow questions explains a verdict better than one broad question, even though the broad question was the more accurate single signal.
  • Designing an agent that moves money is mostly about deciding when it shouldn't.

What's next for Invoice Guard

  • Connecting a mailbox directly, so invoices are checked without forwarding.
  • Testing on real scam invoices, to establish an actual accuracy figure.
  • Handling disputes and refunds for invoices that turn out to be fraud.
  • Proper sign-in, and more than one business per deployment.

Built With

Share this project:

Updates

Submission history