Paypilot

The AI invoice agent that can’t hold your money

Comment

Inspiration

Business email compromise and fake invoices cost companies billions every year. A changed payee email, a look-alike supplier, a line hidden inside an invoice. Putting an AI agent in charge of paying invoices makes this worse, because the agent can be tricked in exactly the same ways. We wanted to find out what it takes to let an agent do the work without ever letting it hold the money.

What it does

PayPilot Shield lets a business owner type "Pay INV-001" in a chat. The agent finds the invoice and asks for the payment, but it never moves money itself. A deterministic rules engine decides one of three outcomes:

  • AUTO_PAY: inside the owner's signed policy and every check passed. The payout is sent through the PayPal Payouts API.
  • NEEDS_APPROVAL: valid, but above the auto limit or a first payment. The owner reviews the exact payment and approves it with a cryptographic signature.
  • BLOCKED: a check failed. No PayPal call is made and the reasons are shown in plain words.

What makes it trustworthy:

  • The owner signs the policy. The spending limit, the monthly budget and the verified-suppliers rule are signed in the browser with a non-extractable ECDSA P-256 key. The key never leaves the browser. No valid signature means no payment.
  • Truth checks on every invoice: unverified supplier, changed payee email, duplicate invoice, amount outlier against the supplier's history, new payee, round-amount pattern, and prompt-injection text hidden inside the invoice.
  • A last gate inside the payout code. Right before PayPal is called, the signed policy, the invoice snapshot, the budget and a replay guard are verified again. No code path can skip the rules, and a tampered policy is refused.
  • Signed supplier trust and an AI screen. Verifying a supplier needs the owner's signature. The AI screen can only raise a request to NEEDS_APPROVAL, never lower its risk.
  • A replayable audit log. Every step of every decision is stored and can be replayed on screen: invoice loaded, history checked, truth check, decision, verifier, payout.
  • Real PayPal events. PayPal's webhook is verified with PayPal's own signature check and settles the payout. The status is read back from PayPal, never trusted from the message.

How we built it

  • Backend: FastAPI, SQLAlchemy and PostgreSQL. PayPal Payouts (sandbox) sends the money, and PayPal webhooks with signature verification confirm it.
  • AI agent: openai/gpt-oss-120b on Groq through an OpenAI-compatible client. The agent has no tool that moves money. Its only way to ask for a payment is submit_payment_request. If the model fails, a rule-based matcher takes over so the product keeps working.
  • Frontend: React, Vite, TypeScript and Tailwind. AG Grid powers the Payments, Invoices and Suppliers screens, and the WebCrypto API does the signing.
  • Deploy: Docker Compose with nginx, HTTPS and security headers including a content security policy.
  • Quality: more than 80 automated tests, including forged webhook signatures, replayed approvals, tampered policies and a race on the monthly budget (a Postgres advisory lock stops two payouts from overspending it). CI runs the tests, a dependency audit, the frontend build and Playwright end-to-end tests.
  • Claude Code was our pair programmer for much of the code, tests and documentation.

The golden rule we built around: the model proposes, deterministic code decides. AI can raise a request's risk level but can never lower it, and invoice text is treated as untrusted data.

Challenges we ran into

  • A payout is not instant. In the sandbox a payout stays pending for about 35 seconds. Polling PayPal in a loop made the agent feel slow and never caught the result. The verified webhook solved both problems.
  • Idempotency. PayPal refuses a repeated batch id, which protects against double payments but broke us after a database reset. The batch id now carries the request creation time.
  • Sandbox quirks. Sandbox accounts in our country were refused for Payouts, while US accounts worked.
  • Models behave differently. One model sent null for an optional field and the provider rejected the whole call. We changed the tool schema and added a test.
  • Prompt injection. Invoice text can say "ignore previous policy". The rules engine scans every text field the supplier controls and blocks it, and the AI cannot override that block.

Accomplishments that we're proud of

  • An agent that is safe by construction, not by prompting: we removed the money tool and kept the decision in plain code.
  • A signed-policy and signed-approval flow that runs entirely in the browser, with a tamper test that stops payments.
  • A real end-to-end run on the live site: an AUTO_PAY invoice paid, an over-limit invoice approved by signature and paid, and a fake invoice with a changed payee email and injected instructions blocked, all visible in the replay.

What we learned

Letting an agent act with money is mostly an authorization problem, not an AI problem. Signed intent, a gate at the point of payment and an audit trail you can replay matter more than the model.

What's next for PayPilot Shield

Fraud thresholds inside the signed policy, supplier edits and un-verify with a signature, dispute handling, injection detection in more languages, and multi-user roles.

Built With

Share this project:

Updates

Submission history