KAVACH-A fraud spike detector

Inspiration

Fraud detection systems in most fintech pipelines treat every transaction the same way: one global model, one static threshold, no memory of who the transaction actually belongs to. That works for catching obvious anomalies, but it misses the most common real-world signal a bank or user would actually notice — "this doesn't look like me." We wanted to build a system that combined a rigorously validated global fraud model with a personalized, explainable layer on top of it, without ever compromising the integrity of the underlying model. JUDGE (Just-in-time Unified Detection and Guarding Engine) is our answer to that: a fraud-spike detection system that scores every transaction against both a trained classifier and the person's own transaction history — and can explain why.

What it does

JUDGE ingests transactions — live-streamed, or uploaded by a user as a CSV, PDF bank statement, or a screenshot of a payment app — and scores each one for fraud risk using an XGBoost classifier trained on real transactional patterns. On top of that score, JUDGE computes a personal baseline for the logged-in user: their typical spend, their usual merchant categories, their usual hours of activity. A transaction that's statistically normal for the population but wildly abnormal for this specific person gets flagged with that context attached, not just a bare number.

Users can:

Sign up and log in via JWT-authenticated sessions
Upload transaction data in whatever format they actually have it — a clean CSV, a real downloaded bank statement PDF, or even a screenshot of a UPI payment
Browse their upload history in a sidebar, batch by batch, the same way you'd browse chat history in a modern AI product
See their personalized deviation from their own baseline, computed as a z-score:

$$z = \frac{x - \mu_u}{\sigma_u}$$

where $x$ is the transaction amount, and $\mu_u$, $\sigma_u$ are the user's own historical mean and standard deviation — never the global population's.

How we built it

We started from an existing loan-underwriting codebase as an architectural baseline, reusing its XGBoost pattern, its Gemini-powered explanation layer, and its dual frontend structure, then rebuilt the pipeline around the Sparkov fraud detection dataset (deliberately rejecting a PCA-anonymized alternative, since PCA-transformed features are fundamentally incompatible with explainability — a fraud analyst can't act on "feature 14 was high").

The core classifier is frozen once validated: ROC-AUC of 0.86, recall of ~79%, with an operating threshold chosen not by convention but by explicitly minimizing a cost function over false negatives and false positives:

$$\text{threshold}^* = \arg\min_\theta \; \left( w_{FN} \cdot FN(\theta) + w_{FP} \cdot FP(\theta) \right)$$

Every feature after that point is designed to be additive, never a modification to this frozen core — personalization, new upload formats, and new explanation context all sit alongside the model's fixed feature vector, never inside it.

We split work across a two-person team using person-wise Git branches (rather than phase-wise), file-ownership boundaries to avoid merge conflicts, and Antigravity as our coding agent — with every implementation plan reviewed for scope creep and regression risk before a single line was written. A 36/36 pytest suite, covering authentication, upload validation, and — most critically — cross-user data isolation, gave us the confidence to keep building on top of what already worked without silently breaking it.

Challenges we ran into

The single biggest challenge was the gap between our training data and reality. Our model's raw schema — card_num, device_id, declined — reflects the synthetic dataset's card-present fraud simulation. Real bank statement exports contain none of these fields, because the concept doesn't apply to a personal UPI/bank statement at all. A naive "just parse the CSV" approach either silently fabricated data or rejected every real file outright.

We solved this with a strict two-step pipeline: an extract step that detects the real header row (skipping bank-name and account-summary preamble rows), maps real column names to our schema via a synonym lookup, and explicitly labels any field it can't populate — never guessing a plausible-looking fake value — followed by a mandatory confirm step before anything is scored or persisted. Testing against a real personal bank statement even surfaced a subtler bug: trailing footer/total rows being misparsed as transactions, caught only because the two-step design meant we could inspect the extracted preview before it was ever scored.

We also learned, the hard way, that "additive, not modifying" has to apply architecturally, not just to the model — a heuristic parser refactor that seemed purely internal to one file risked silently changing the behavior of an already-tested endpoint, a class of regression that's easy to introduce with good intentions and a coding agent that doesn't know your test suite exists until you tell it.

Accomplishments that we're proud of

A fraud model whose threshold was chosen deliberately, via cost minimization, not guesswork
A personalization layer that adds real explainability without ever touching the frozen model
A real-world statement parser that handles messy, actual bank exports — not just our synthetic training format — while refusing to silently fabricate data it can't honestly extract
Zero regressions across six feature phases, verified by an automated test suite we can actually trust

What we learned

Explainability isn't a feature you add at the end — it's a constraint that has to shape every architectural decision from the schema you choose to train on, to how you compute a threshold, to how you handle a personal baseline. We also learned that "additive only" is a discipline that has to be actively defended in every implementation plan, not an assumption you can make once and forget.

What's next for JUDGE

Extending the personal baseline into a richer behavioral profile, adding OCR-assisted extraction for scanned (image-based) statement PDFs, and building out a fully polished upload-confirmation UI so the extract → confirm flow we built for real-world data is as smooth for a user as it already is for our test suite.