PayPause

What's new: the verdict comes from your payment history, not just from the email.

Blind test of the current rules: 20 everyday emails written after the rules were frozen (by a separate AI session that saw only the café's payment records): 10/10 frauds held, 0/10 honest emails held (n = 10 each, 95% ≈ 72–100%), 2/10 sent for a quick check. Same catch rate as asking the model directly, fewer false alarms (2 vs 4 of 10, within noise), and it tells you which number to call.

Try it in 30 seconds: open the link → Next → pick "Dairy, new remittance" → Check this email. Sample chips replay recorded Gemma 4 readings (no key needed). Your own emails need a free key from aistudio.google.com/apikey — without one they run in a deliberately cautious keyword mode that sends most payment emails to CHECK. Your records never leave the browser; only the email text goes to the model.

Inspiration

For the owner or bookkeeper who approves vendor payments with no security team. Business email compromise — "a scam targeting businesses or individuals working with suppliers and/or businesses regularly performing wire transfer payments" — cost $3.05 billion in complaints to the FBI in 2025 (IC3 2025 Annual Report, pp. 59, 9, 26), and "chat generators can quickly create official-sounding emails" (p. 39). The typos are gone. A small business pays by bank transfer with no second check. The check that actually stops this — call the vendor on a number you already have — should happen every time.

What it does

1. Your records — a CSV of past payments (minimum: vendor and payee email; account, amount and phone sharpen the checks).
2. The email — paste it, headers included.
3. The answer — HOLD / CHECK / CLEAR with every red flag explained against your records, and a callback script that dials the number from your records — never one from the email.

How we built it

• Gemma 4 reads; a checklist decides. Gemma 4 26B-A4B (open weights, via the Gemini API; JSON schema, temperature 0) turns the email into a fixed form and, in a parallel call, gives its own second opinion — which can add a flag, never remove one.
• Checks that don't depend on the model: lookalike/subdomain senders, Reply-To, failed Authentication-Results, and a plain-text backstop for destination accounts, gift-card requests and bank-change phrases.
• Falls back loudly: no key or a model failure → keyword reader, labelled on screen, which never shows a bare CLEAR for a payment request.
• Runs in the browser. Clean Architecture with a layer check, 60 unit tests, contract tests for both model calls, a real-browser smoke test.

How well it works

Three blind sets, each written by a separate AI session (same model family as our coding assistant; not real mail), committed unopened and first run after the rules were frozen. "Stopped" = HOLD or CHECK.

• Current rules (20 everyday emails, writer saw only the records; commit 29211ae, tag set-records2): 10/10 frauds held, 0/10 honest held, 2/10 checked. Gemma asked directly: 10/10 stopped, 4/10 honest interrupted — within noise of PayPause.
• Reading meaning (20 emails dodging keywords, 8 not in English): 10/10 stopped, 1/10 honest checked — Gemma asked directly does as well, so the model is doing the reading.
• Do records matter? (16 routine-looking emails): the blind run stopped 5/8 and found a bug (multi-line invoice footers); after the fix 8/8 — not blind. Gemma alone interrupted 5/8 honest emails; with a records summary in its prompt it stopped 4/8. The second opinion changed nothing on the three blind sets; across all eight sets it added one stopped fraud and one false HOLD. Small sets (10/10 ≈ 72–100%). Every email and verdict: https://github.com/Ryugi62/paypause/blob/main/docs/eval.md

Challenges we ran into

Five tagged rule versions (v3–v7); three fresh, separately written blind sets were run right after a tag — including the one that exposed the footer bug above, which we kept in the results. Free-tier quotas ran out mid-evaluation, so we moved to Gemma 4 and separated live recording from offline, reproducible evaluation.

What's next

Read PDF attachments; run Gemma 4 locally so email text never leaves the machine; import directly from accounting software; a Gmail/Outlook add-on.

What works and what doesn't

Works: the three-step flow, samples without a key, live reading with a key. Limits: login phishing (out of scope), details only inside attachments, pretexting before any payment is asked; announced bank changes always get a CHECK by design; one honest Spanish invoice was misread as a change and got a CHECK.

Built during ForgeHacks; tools disclosed

All code written during the event (first commit 2026-10-05). An AI coding assistant (Claude) helped write the code and the dev email set; the test sets were written by separate AI sessions (same model family) from https://github.com/Ryugi62/paypause/blob/main/data/eval/BRIEF.md. Narration: Microsoft Edge neural TTS. Data is fictional.

Built With

• gemini-api
• gemma
• github
• playwright
• typescript
• vite
• vitest