ComplianceFroge

Landing Page - Problem Statement
Circular Ingestion and Updation
Transaction queue
Orchestrator agent with Anti-money landering detection sub-agents
Flagged areas with reasoning and the required regulations
Areas where the transaction proved to be not anomalous with sufficient evidence
Breakdown of the outputs from each layer
The suspicious transaction report in the legal format with citations that can be linked to their legitimate sources
The problem in todays AML techniques
Our solution

Inspiration

Our team had started to explore the world of Finance and the problems that plagued it. One of the most prominent issues seemed to be about companies struggling with regulatory compliance especially when it came to the Financial regulatory Bodies in the country. This was specifically when it came to suspicious transactions- people were unable to file the report before the deadline, they were overloaded by the sheer volume of flagged false positives and the ever-changing rules and regulations made it incredibly difficult to stay in compliance. But what's even worse would be reusing old verdicts under superseded circulars. This is why a dynamic approach to the way the system's memory stored, viewed and used circulars was necessary so that the verdicts of the past expire the second that regulation did.

What it does

The orchestrating agent fires a combination of specialized AML sub-agents in parallel - velocity and structuring, sanctions screening, mule accounts etc based on inferences drawn from the transaction itself. This produces a suspicion score with ample reasoning, both are vital for future stages.
Every completed case is properly stored, this is to ensure that if a similar case shows up and no changes have been made to the regulation used, it can short-circuit the chain and directly use insights gained from past cases to reach a proper conclusion.
The memory chunk is retired as soon as the circular isnt relevant anymore: All verdicts are tied to the version of the various rules and regulations that make up our corpus.
The system is not a black box: It provides proper verifiable reasoning, the applicable circulars that all trace back to their original sources and a confidence and suspicion score. This, in addition to an agent-wise explanation of each type of AML check.
All inconclusive cases with a low confidence score are sent to the human analyst with all the outputs from prior layers attached. This is to ease the entire process and ensure the SLM can improve as well.
The final output is a ready to submit Suspicious Transaction Report that fully matches the approved legal format.

How we built it

The most important component of the system was ensuring that the data was sound, even prompting only helps to a certain extent. Thus we settled on a vector RAG - AI search system, where all rules and regulations are ingested from the Master Directions of the financial bodies, with scanned circulars run through Amazon Textract and the raw documents archived in Amazon S3.
These circulars are then chunked, embedded and indexed into CockroachDB as native 768-dimension vectors with a distributed vector index. An AI search is used to surface the most relevant rules and regulations.
An orchestrator agent uses all the information it has from a transaction, ingested through Amazon SQS, to fire a combination of 6 specialized AML sub-agents (e.g. Structuring and velocity errors, Mule accounts, Geolocation anomalies etc). Based on the algorithms used and reasoning from the SLM, run locally on Ollama through a provider-agnostic model gateway, a suspicion score is produced which is sent to the next layer: Report generation.
The report generator cross references the detected type of fraud to its corresponding rules and regulations from the AI search and reasons as to which of the top searches will actually satisfy all the necessary conditions. Now there may always be issues around hallucinations and the SLM not producing sound reasoning, which is what the next layer is built to catch.
What was extremely helpful here was a maker-checker: An SLM whose primary role is to interrogate the previous layer and ensure its accuracy and sound reasoning. There is a feedback loop until the desired conclusion is reached. In addition to this, if it is unable to come to the required conclusion, the packet is once again sent to the human analyst. The final Suspicious Transaction Report is written to Amazon S3.
All completed cases are stored in CockroachDB as MinHash feature sets along with the final verdict and the regulation version that accompanied it. If a similar transaction appears in the future, the SLM can circumvent the entire process and directly come to an informed decision based on past cases. The regulatory watch job that keeps this corpus current runs on AWS Lambda via EventBridge.

Challenges we ran into

Ensuring the database was always updated posed an issue initially. Scraping thousands of documents across all the websites and comparing them to the database wasn't really feasible. Instead we settled on scraping the notifications and withdrawn circulars pages: less compute and much more efficient as a system since the notification page is the origin of most stimuli and signals for our ingester. We also had to ensure that if a cache check skipped processing layers underneath, it had to be provably still valid under the current regulation. The hash update and cache invalidation occur in the same transaction to make sure no stale verdicts reside in memory.

Accomplishments that we're proud of

Creating a cohesive system that gets around most of the issues faced when keeping up with today's rules:

Memory that is tied to the version of the regulation it was decided under, so a verdict retires the moment the circular behind it does- constant updation ensures constant compliance.
Leaning on human analysts for inconclusive cases and actually providing ample material for them to come to an informed decision - a system that actually answers the why.
Running the entire project on small language models with a provider-agnostic gateway, so the same pipeline runs locally, on AWS, or fully on-premise for institutions that legally cannot send transaction data off their own infrastructure. Techniques around prompting and grounding are what ensure accurate results, not model size.
Creating a dynamic system that changes with rules and regulations and can be added as a layer on top of older systems if phasing out prior tech is not possible.

What we learned

Compliance is multi-faceted, there is not one answer for all. However, you can cover a very large chunk of fraud and anti-money laundering checks by efficiently using algorithms and SLM reasoning.
The hard part of memory is not storing or retrieving it, it is knowing when something you remember has stopped being true. Most of our design ended up being about invalidation rather than lookup.
Grounding and proper prompts with the necessary context does wonders for a system. A model is only as good as the information it is provided with. This was definitely drilled in throughout the entire process.
Agentic systems solve a lot of issues in prior systems and this is a concept with a lot of potential and definitely something that we will be exploring further, out of the scope of this project as well. The maker-checker was proof of this for us - a second model interrogating the first caught unsound reasoning that no amount of prompting the first model ever fixed.
Isolation level turned out to be an architectural decision and not a tuning knob. A good part of the audit chain is only correct because the database is serializable by default, which is not something we expected going in.

What's next for ComplianceForge

Our goal is to create a system for scale. The next step there is going multi-region, so that transaction rows are pinned to Indian regions at the schema level rather than in a deployment document, and the system survives losing an entire region. Beyond that, ironing out all the kinks and making the system fully usable and not just an MVP would be the next goal, along with adding more features and testing the possibilities of our system. We also want to use changefeeds so that a filed report triggers the downstream alerting as part of the same commit. Also checking if there is a certain commercial angle we can modify the same system for, with respect to individual organisations and their rules and regulations.