Fraud Forensics by Claudius Maximus

Inspiration

The Infosys track asked for something forensic auditors already know from experience: fraud almost never announces itself as a single anomaly. A phantom vendor doesn't look wrong in isolation — it looks wrong when you notice the same purchasing approver signed off on their own vendor's invoices. We wanted to build something that reasons like an actual investigator: start from a statistical hunch, go verify it against the real books, and only accuse when the evidence survives scrutiny. That's a very different shape than "run a classifier and print the top-scoring row," and it's what pulled us toward a multi-stage pipeline instead of a single model call.

What it does

Fraud Forensics takes a company's accounting data (ledger, invoices, bank transfers, vendors, purchase orders, contracts, employees — a SQLite "estate") and produces a court-ready case file identifying five scheme types: phantom vendors, kickbacks, round-tripping, threshold splitting, and revenue inflation.

The pipeline runs in five stages:

Deterministic detectors + a CART classifier rank every entity into a prioritized list of "leads" — a statistical hint, never an accusation.
An LLM investigator (Gemma 3, running locally via Ollama) gets each lead and a toolbox to query the real database — it can pull invoices, trace transfers, check purchase-order approvers — and forms a hypothesis from what it actually reads, never inventing a record.
A deterministic validator re-checks every cited record against SQLite and reconciles the accused peso amount against real evidence within 2%. Nothing prints without passing this gate.
A challenger LLM stress-tests survivors, actively looking for an innocent explanation before a finding is finalized.
A React frontend renders the resulting findings, the money trail between entities, and the investigation timeline — with Excel upload for auditors to bring their own company data.

How we built it

Backend in Python: SQLite for data, a self-contained JSON-exported decision tree (no sklearn dependency at runtime — the runtime shouldn't require the exact scikit-learn version it was trained with), and Ollama serving Gemma 3 12B locally so the whole thing replicates deterministically with zero network calls and zero per-token billing. The frontend is TypeScript/React (Vite, Tailwind, shadcn/ui, Zod) sharing a strict schema contract with the backend's JSON output, so a submission either validates or the UI knows exactly why it didn't.

Two people built genuinely separate halves — the forensic pipeline on one branch, the UI on another — and merging them back together late was its own exercise in keeping both halves honest.

Challenges we ran into

Ollama's native tool-calling doesn't support Gemma 3. We didn't discover this until every LLM-backed run silently produced zero findings. Fix: describe tools in the prompt and have the model request them as plain JSON — the same trick used before native function-calling existed.
A model reasoning in Spanish translated an English enum value (bank_txns → "transferencias") when citing evidence, and the validator correctly rejected it — but that meant a true finding was getting thrown away for a translation slip, not a data problem. Fixed by listing the literal enum values, untranslated, directly in the prompt.
Peso reconciliation double-counted an invoice and the bank transfer that paid it as two separate amounts instead of the same money seen twice — a bug that would pass our own checks but fail the official judge's validator.
Silent context-window truncation. A multi-turn tool-calling loop can fill a 4096-token context by turn five or six, cutting the model's JSON answer off mid-sentence with no error — we had to request a larger context window explicitly rather than trust the default.
Merging two branches that hadn't touched each other's territory in weeks — with one file on the "main" side that turned out to be broken, unfinished, and never actually exercised by its own test suite.

Accomplishments that we're proud of

Every business rule — the peso tolerance, the exhibit minimum, the scheme-type enum — lives in code, never buried in a prompt where nobody could audit it. No finding reaches the final report without a deterministic gate confirming every cited record actually exists and every peso reconciles. The whole system runs 100% offline against a local model, with a full regression suite that catches each of these bugs the moment they'd reappear — several of which we found and fixed live, mid-hackathon, because the tests told us something was wrong before a judge would have.

What we learned

Local open-weight models need fundamentally different handling than hosted commercial APIs — no native tool-calling, tighter context budgets, and a real need to constrain decoding (format=json) rather than hope for well-formed output. We also relearned a forensic-audit-shaped lesson in software form: separating "what the model believes" from "what can be verified" isn't extra caution, it's the entire point — an LLM's confidence is not evidence, and the only thing that should print is what survives a second, deterministic look.

What's next for Fraud Forensics

Wire the full per-step event stream (tool calls, hypotheses, challenges) into the frontend's live timeline instead of just start/finish metrics; run against a sealed holdout estate we've never tuned against to get an honest generalization number; and extend the scheme-type library beyond the five official types toward whatever real accounting data throws at it next.