ProofBeforePay

Paste a job offer or client message. Nemotron agents check the company, its domains, links and every claim, then a second model tries to prove the verdict wrong. Proof, not a score.

Comment

Inspiration

Fake job offers are everywhere right now. The FTC says reported losses to job scams went from $90 million in 2020 to $501 million in 2024, and the number of reports nearly tripled. The people being targeted are usually the ones who can least afford it: students, people between jobs, freelancers waiting on their next client.

What struck me is how checkable most of these scams are. The "recruiter" writes from a domain registered last week. The company's own careers page says it never asks candidates for money. The exact same message has already been posted on a scam-report forum. The evidence is out there. But nobody has the time or the know-how to dig through it at 11pm, when the offer "expires tomorrow".

I didn't want to build another scam detector that spits out a score. Most offers people receive are real, and telling someone a genuine job is a scam can cost them that job. So the question ProofBeforePay answers is narrower and more useful: should I trust this enough to take the next step?

What it does

You paste the message, plus the sender's email if you have it. Most of these scams arrive on WhatsApp or Telegram, so you can also just drop in a screenshot of the chat. ProofBeforePay runs an investigation you can watch live, and in about forty seconds you get one of three answers: Proceed, Verify first or Stop.

Every answer comes with its reasons:

  • Hard signals found by code. Upfront fees, the fake-check-for-equipment scheme, gift cards and crypto, interviews that only happen over chat, pressure tactics, and text that tries to give instructions to AI checkers. It reads the domain registry (RDAP) for every domain in the message, compares the sender with the company's real website, and opens every link through Tavily to see whether the page borrows the company's name or asks for a payment, a password or ID.
  • Claim-by-claim checks. "Jane is a recruiter at Acme." "Acme is hiring for this role." Each claim is marked verified, contradicted or inconclusive, with a quote from the page it came from.
  • A verdict that has been argued with. After Nemotron Ultra decides, Nemotron Super runs its own searches trying to prove it wrong, checks each of Ultra's findings against the sources it cites, and lists exactly what evidence would change the answer. On a genuine Canonical recruiter email, for example, it says the one thing still worth checking is whether the email passes SPF and DKIM, and how to check that yourself.
  • The Proof Trail. An interactive graph from sources to claims and signals to findings to the verdict. Click Amazon's own fraud page and you can see it contradict the "refundable onboarding fee", which feeds two findings, which lead to Stop.
  • What to do next. What not to do yet, questions to ask the sender, and a polite reply that asks them to prove who they are without sharing anything about you.
  • Ask the report. Follow-up questions like "Is the pay realistic?" or "What should I check first?" get a short answer from that report's evidence only, with citations you can click.

Finished investigations get a shareable link, can be replayed step by step, and can be downloaded as Markdown or saved as a PDF. With a free account, every check is saved to a dashboard that shows exactly how the agents worked on it.

How I built it

The backend is a FastAPI app running a seven-step pipeline with asyncio. Every step writes to an event log, and the browser follows along over Server-Sent Events, so you see signals appear and claims flip from "checking" to "contradicted" as it happens. Finished runs, the evidence trail and a spending ledger are stored in Supabase.

NVIDIA Nemotron on Nebius Token Factory. I used three Nemotron models and gave each the work that fits its size:

  • Nano reads the message, does the first pass on every claim (in parallel), and writes the brief and the reply.
  • Super only gets a claim when Nano is unsure or can't verify it, and it gets a fresh search to work with. It's also the challenger: it plans counter-searches, argues against Ultra's verdict and audits its findings. Using a different model for that matters, because a model checking its own work tends to agree with itself.
  • Ultra is called once per investigation. It reads every signal, claim check and source in one long reasoning pass, and that reasoning streams into the UI live.

Super also answers follow-up questions about a finished report. It only sees that report's evidence, and code removes any citation that points to something the report doesn't contain. For screenshots, a small vision model on Token Factory (MiniCPM-V) only copies out the text. You check that text before anything runs, and the Nemotron models do all the reasoning.

Token Factory's API is OpenAI-compatible, so I used the official openai SDK with streaming and usage reporting on every call. Each call is metered: model, tokens, latency, cost, and what the same tokens would have cost on Ultra. Every report shows that comparison. Over ten live runs of the full pipeline, a check cost about two cents and routing came out 52 to 62% cheaper than sending everything to Ultra.

Some decisions are made by code, not by the models. Code drops citations to sources that don't exist and quotes that can't be found in their source. It refuses a Proceed when money, credentials or ID are requested. And it decides what the challenge can change: if Ultra and Super disagree, the answer becomes Verify first, unless the message asks for money, in which case a Stop stays a Stop.

Tavily does the web side. Searches find the official site, fraud warnings and the recruiter's footprint. An exact_match search looks for the message's most distinctive sentence, because scam scripts get reused word for word. crawl with plain-English instructions walks the company's careers pages looking for the role. extract reads the company's own fraud-warning page in full and opens suspicious links, so my server never touches them.

The frontend is Next.js 16 and TypeScript, exported as static files and served by the same FastAPI container. GSAP and Lenis handle the scroll storytelling, there's a small React Three Fiber scene behind the hero, and the Proof Trail is a measured SVG layer over plain HTML nodes. Since the demo is public and runs on my own card, the server has a hard spending ceiling: each check reserves a worst-case amount before it starts, and live checks pause before the limit while the recorded examples keep working.

Accounts are built in: sign-up with a six-digit code sent through Resend, scrypt-hashed passwords, and sessions stored only as a digest. Signed-in users get a dashboard where all seven agents sit on one live rail, showing each step's model calls, web calls, tokens and cost, plus their history and a side-by-side compare. An admin dashboard shows the live health of Token Factory, Tavily, Supabase and Resend, the model budget, every user, and all feedback and messages.

There are 57 tests covering the pipeline, the guardrails, the challenge rules, the signals, the API, the account flows, asking a report and the screenshot checks. They use fake models, a fake web and an in-memory database, so they run offline and cost nothing.

Challenges I ran into

  • Getting the models to be honest about not knowing. It's easy for a model to call a claim "contradicted" when the search just came back empty, and that's exactly the mistake that costs someone a real job. The rules say missing evidence is always inconclusive, and code throws away any quote that can't be found word for word in the cited page.
  • The first genuine email I tested got flagged. It was a normal recruiter email from a real company domain, and the report marked one claim as contradicted because an interview-prep blog described the hiring steps a little differently. Then it called that blog the company's own handbook. I tightened the rules: small differences aren't contradictions, a third-party blog can't overrule a company about its own process, and sources have to be named for what they are.
  • Ultra kept reading more into a snippet than it said. A company's warning said scammers use "corporate-style emails", and Ultra turned that into "scammers use the company's real email domain", which is the opposite of what the full page says. Rewording the prompt didn't fix it. Reading the company's own warning page in full fixed half of it, and having Super audit each finding against its sources catches the rest.
  • Made-up names belong to real people. For one test I invented a recruiter name, and it turned out a real person with that name works at that company. Since then, prompts forbid describing people who merely share the sender's name, and the genuine-email example is signed by a team rather than a person.
  • Careers pages don't live where you'd expect. Amazon's jobs are on amazon.jobs, not amazon.com. And my official-domain matcher once decided a different design studio was the official site because both names contained "studio". Generic words no longer count.
  • Streaming reasoning cleanly. Reasoning can arrive as a separate field or as <think> tags split across chunks. I wrote a small splitter so the UI shows the reasoning live without mixing it into the final JSON.

Accomplishments that I'm proud of

  • Every conclusion can be traced back to a registry record, a quote from a real page, or a pattern in the message, and the Proof Trail lets you click through it.
  • Every verdict has to survive a second model trying to prove it wrong, and you can see what it tried.
  • It says yes too. A genuine recruiter email gets a Proceed, with the one thing still worth checking.
  • The cost routing measures itself. The savings aren't a slide claim, they're in every report.
  • It treats everything it reads as untrusted: messages that try to instruct the AI are flagged and can never get a Proceed, and suspicious links are never opened by my server.

What I learned

  • Small models can handle narrow jobs. Give Nano one clear task at a time and a way to hand off when it's unsure, and Ultra only needs to run once per investigation.
  • A second model is a cheap, effective check on the first. The challenge costs about a fifth of a cent and caught errors that prompt changes didn't.
  • A lot of verification is really a search problem. Searching for one exact sentence, crawling one company's careers pages or reading its own warning page gives the model something concrete to judge instead of asking it to remember.
  • "Inconclusive" is a valid answer. Building that into the prompts, the code and the UI made the whole thing more trustworthy.

What's next

  • Checking messages where they arrive: a browser extension for LinkedIn and email, and a forwarding address.
  • Reading email headers (SPF, DKIM, DMARC) when people paste the original email, since that's often the deciding question for a genuine-looking message.
  • More languages. The safe reply already comes back in the language of the original message.
  • A shared, privacy-safe list of confirmed scam scripts, so one person's report protects the next person.

Built With

Share this project:

Updates

Submission history