Smurf

SMURF

Inspiration

SMURF started with a question: when AI flags a suspicious money flow, what should an investigator trust?

Finding unusual activity is only part of the problem. Investigators also need to understand which transactions support an alert, whether ordinary business could explain the pattern, and which uncertain cases deserve another look.

We wanted to make those decisions visible: follow the money, inspect the evidence, compare model judgments, and measure the mistakes.

What it does

SMURF is an investigation and evaluation prototype that helps users triage suspicious transaction patterns.

Its visual workflow has four stages:

Search: A first-pass model reviews transaction neighborhoods using transfer rows, computed graph facts, and retrieved pattern descriptions.
Ambiguous: Low-confidence alerts and cases with substantial model disagreement enter a review tray.
Look closer: A stronger model reassesses those selected cases.
Settled: Users inspect the resulting classifications, supporting transfers, false alarms, and missed groups.

The interface lets users explore account connections, follow transaction paths, compare models, and export case evidence. Human investigators retain the final decision.

How we built it

We used IBM’s synthetic AMLworld HI-Small transaction dataset, starting with a slice of 2,824 transfers across 1,797 accounts. We constructed 60 evaluation neighborhoods: 20 containing IBM-labeled laundering groups and 40 containing ordinary transfers.

Our Python backend computes facts such as account connectivity, directed loops, transaction bursts, and rapid forwarding between accounts. These facts give models explicit evidence to assess alongside the original rows.

We accessed multiple model providers through Nicrron, testing them on consistent inputs and requiring structured answers with a verdict, pattern, flagged transfer IDs, confidence, and explanation. We also compared raw-row prompts with versions that included computed facts and knowledge-base retrieval.

Our scoring pipeline checks flagged transfer IDs against IBM’s labels and measures precision, recall, F1, group coverage, false alarms, latency, and estimated cost. We evaluated additional slices with accounts separate from the original slice.

We integrated PRISM by Block Convey for session traces, tool steps, retrieval records, and evaluator runs. The frontend uses HTML, CSS, and JavaScript to visualize the network and replay recorded model investigations without depending on live API calls during the demo.

Challenges we faced

Ordinary activity can resemble suspicious structure. Payroll, merchant collections, and settlement activity can produce busy accounts, bursts, and forwarding chains. Recognizing a shape does not establish criminal activity.

The dataset contains shortcuts. Every laundering transfer in our working slice uses ACH. We tested models with payment format hidden to examine how much their performance depended on that field.

Better wording did not automatically produce better results. Some stricter prompts reduced false alarms while missing more labeled groups. We learned to evaluate prompt changes rather than judge them by how convincing they sounded.

Model behavior and infrastructure varied. Providers differed in latency, output formatting, token requirements, and availability. We had to distinguish technical failures from incorrect classifications while managing the evaluation budget.

Accomplishments we’re proud of

We built an inspectable workflow connecting transaction evidence, model decisions, evaluation results, and investigation traces.

In one recorded second-pass experiment, the stronger model reviewed 30 unresolved neighborhoods, identified two additional IBM-labeled groups, and raised no false alarms among the 23 ordinary neighborhoods in that tray. This is a result from that specific synthetic sample, rather than a claim about performance at a real bank.

We also made missed groups visible in the interface. Showing where the system fails is essential to understanding where it can help.

What we learned

A useful model must do more than recognize a suspicious shape: it must identify the right transfers without sweeping ordinary activity into the alert.

We learned that model disagreement can help select cases for review, that confidence needs empirical checking, and that some labeled transfers lack enough observable context for reliable classification.

Our prototype evaluates triage and model reliability on synthetic data. Its curated neighborhoods are enriched with laundering examples, so the results do not establish real-world detection performance.

What’s next for SMURF

We want to generate candidate neighborhoods without using answer labels, add richer account history, and expand evaluations across larger and more representative samples. We also plan to persist investigator feedback and use measured performance to improve when cases receive deeper analysis.

Data credit: IBM Transactions for Anti Money Laundering, HI-Small, by Erik Altman et al., shared under CDLA-Sharing-1.0. All transactions used in this prototype are synthetic.