Inspiration
In India, a bank's Principal Officer decides for every AML alert whether to file a Suspicious Transaction Report (STR) with FIU-IND or close it, and must be able to defend either choice at inspection, sometimes months later. FIU-IND received 4,33,761 STRs in FY 2021-22 and describes STR quality as something it works on with banks (FIU-IND Annual Report 2021-22, §3.2.1). The part that has to survive scrutiny is the written reasoning and the numbers in it. A language model that adds up figures itself and sounds certain is the wrong tool for that job. So we built a copilot where the model can reason and write, but cannot be the source of a single rupee.
What it does
- Twin Cases. Two synthetic alerts fire the same rules on the same ₹3,45,000. One is a mule account; the other is a family paying a hospital bill. Side by side, the calculator finds the one signal that differs (a flagged counterparty): the filing rests on exactly one fact, and the closure comes with three conditions that would reopen it.
- Both readings, before any AI. Every alert opens on what a FILE would rest on and what would reopen a closure. This is computed by re-running a fixed, hashed calculator on every counter-hypothesis (32 runs for ALERT-01, no model involved). The officer decides; the evidence and hashes are one click away.
- Every rupee cited. Any figure in the Ground of Suspicion must equal a transaction amount or a calculator fact, however it is written (₹, Rs., INR, lakh, crore). Formally, for every figure $x$ in the narrative:
$$x \in A_{\text{txn}} \cup A_{\text{computed}}$$
Anything else is blocked, and checked again at the moment of recording.
- An agent that cannot invent evidence. Nemotron Super investigates with tools. If its submission cites a number, fact or rule the tools never returned, it is rejected and sent back.
- A ledger you can re-check. Decisions go to an append-only ledger where each row is chained to the one before:
$$h_i = \mathrm{SHA256}\big(\mathrm{canon}(r_i) \,|\, h_{i-1}\big)$$
so removing or editing a row is detectable. The ledger exports as a package anyone can verify offline. Requests to tip off the customer are refused before any model call.
How we built it
- Nebius Token Factory + NVIDIA Nemotron, in three tiers: Nano 30B (quality check, stress-test memo), Super 120B (11-factor assessment, tool-calling agent) and Ultra 550B (Ground of Suspicion draft). Each step asks for a tier, never a model ID, and the ledger records which model answered. The app's Platform trace shows the tier, the model Token Factory reported, and the tokens and seconds for every call.
- Structured output:
json_objectmode, local schema validation, one repair retry, and fail-closed behaviour. - Calculator of record: a standard-library-only Python program run in an isolated subprocess, with the SHA-256 of its code, input and output recorded for every run. Counter-hypotheses are branch runs of the same program.
- Tavily: typology context (no customer data leaves the bank) and a regulator-only search (RBI, FIU-IND, SEBI and others). If a search fails, the app says it "did not run", never "nothing found".
- Streamlit UI, SQLite with database triggers that make the decision and compute tables append-only, and 451 offline tests plus a failure-mode eval harness.
Challenges we ran into
- Reasoning tokens eat the output budget. Nemotron's hidden reasoning counts against
max_tokens. With small caps, the 11-factor assessment was cut off mid-JSON, and the Ultra draft once returned its own half-finished reasoning ("~209 words… Good.") instead of a narrative. We moved to one generous budget for every call. A reply cut off at the limit is retried once at a higher ceiling and otherwise reported as an error. It is never shown as an answer. - Our first figure gate was too narrow. It only read
₹<digits>, so "Rs.", "INR" and "lakh" spellings of an invented total got through. We replaced it with an exact parser and re-check at recording. - Token Factory Sandboxes. We wrote the backend against the real SDK, but our account was refused access (HTTP 403). The calculator runs locally in an isolated subprocess, and the app says so.
Accomplishments that we're proud of
- A decision's dependencies are computed, not narrated: "this FILE rests on exactly one fact".
- A model-written number cannot reach the ledger, in any spelling.
- On a 6-case holdout (author-labelled, 2026-10-21), the same model drafting without our controls put unsupported figures in 6/6 drafts. With the product, the drafter still did so in 3/6, the gate rejected all 3, and 0 unsupported figures reached an accepted draft. Six cases support no general claim, but they show where the safety comes from.
- In a live run over 16 synthetic alerts (2026-10-21), the agent matched all 12 clear-cut cases (9 FILE, 3 NOT_FILE). An always-FILE rule would score 9/12.
- Every public number traces to a committed, dated report.
What we learned
Trust boundaries matter more than prompts. The drafter still produced unsupported figures in half the holdout drafts even with careful instructions; the deterministic gate is what kept them out. With reasoning models, the reliability of what surrounds the model (budgets, validation, fail-closed paths) matters as much as the prompt.
What's next for Twin trave
- Run the calculator in Token Factory Sandboxes once access is granted.
- Richer seed transactions: most alerts are one summary row today, so only 1 of 9 gold-FILE alerts reaches a HIGH modeled level.
- A Nemotron content-safety layer for paraphrased tipping-off (the current gate is regex-based).
Limitations. All data is synthetic. Twin Trace is not "compliant" and not FIU-IND approved: it helps write defensible decisions, it does not make them. The ledger is tamper-evident, not tamper-proof, and the officer ID is self-asserted. The checklist and stress-test signals are product heuristics, not FIU-IND criteria.
Log in or sign up for Devpost to join the conversation.