Breakwater

Don't let ledger breaks sink your ops. Breakwater uses a multi-agent AI swarm to autonomously reconcile enterprise finances with zero-trust safety.

Comment

Breakwater: Multi-Agent Financial Reconciliation Engine

Inspiration

My inspiration for Breakwater comes from my experience working around international banking, insurance, and large trading firms, where I repeatedly saw the same operational pattern: transactions move through multiple systems, and a small percentage inevitably become "breaks."

These breaks can come from timing differences, FX rounding, processing fees, missing entries, or differences in how two systems record the same transaction.

The matching itself is usually not the hardest part. The real cost comes afterward: operations teams spending significant time and manpower investigating, explaining, documenting, and resolving these exceptions.

Having seen how important — and repetitive — this work is in enterprise financial operations, I started thinking about what an agentic system could do differently.

What if the last 10% of reconciliation could have its own AI operations team?

That became Breakwater.

I wanted to explore whether AI agents could turn a traditionally manual exception-handling process into a continuous, intelligent, and auditable workflow, while keeping humans in control whenever the system is uncertain.

What it does

Breakwater turns financial reconciliation exceptions into a governed agentic workflow.

It starts with deterministic matching, then applies specialized AI only to the records that need investigation.

Deterministic first. Agentic second. Human when uncertain.

Deterministic Matching

Breakwater uses Pandas to reconcile the two feeds using transaction identifiers, then evaluates amount and timestamp differences and identifies missing records.

The goal is to clear the straightforward population without making unnecessary AI calls.

Agentic Investigation

The remaining discrepancies become breaks and enter the asynchronous agent workflow.

  • RAG Memory: text-embedding-005 searches historical resolved cases for useful precedent. A 0.92 similarity threshold is used for near-duplicate detection, while 0.88 is used for historical precedent/context.
  • Investigator: Gemini analyzes the break, identifies the likely root cause, and produces a confidence score.
  • Severity Tagger: Gemma classifies the business severity as low, medium, or high.
  • Resolver: Gemini applies the resolution policy and generates the accounting narrative.

Safe-by-Default Resolution

A break is only eligible for automatic resolution when it satisfies the policy:

known-safe break type + confidence >= 0.60 + severity != high

Only then can a break become auto_resolved.

Otherwise, it is escalated for human review.

This creates an important boundary:

The agents can investigate broadly, but autonomous resolution is policy-gated.

Human-in-the-Loop

Escalated cases appear in the dashboard, where an operator can inspect the investigation, severity, confidence, and narrative before approving the resolution.

The human decision updates the current workflow state and is also recorded as a separate audit event.

Audit and Export

Breakwater separates:

workflow_active — current operational state

audit_events — application-enforced append-only decision history

The current reconciliation state can also be exported as a CSV report for operational and compliance workflows.

How we built it

I built Breakwater as a lightweight cloud-native application using Python and Google Cloud.

Technology Stack

  • Python 3.11
  • FastAPI — backend API
  • Pandas — deterministic reconciliation
  • HTML / Vanilla JavaScript — operational dashboard
  • Chart.js — reconciliation analytics
  • Google Cloud Run — containerized application runtime
  • Google Cloud Firestore — workflow state and audit events
  • Vertex AI — Gemini, Gemma, and embeddings
  • Google GenAI SDK — model access
  • Google ADK — agent/workflow structure
  • Docker, Cloud Build, Artifact Registry — deployment

Specialized Agent Design

Instead of one large prompt, Breakwater divides the work into specialized responsibilities:

Matcher → RAG Memory → Investigator → Severity Tagger → Resolver → Human Review

Each stage has a focused purpose and constrained output.

This makes the workflow easier to reason about, debug, and govern.

Asynchronous Execution

The unresolved break population is processed using FastAPI background processing and:

ThreadPoolExecutor(max_workers=10)

Each break can be investigated independently, allowing multiple exceptions to move through the agent workflow concurrently.

The implementation also introduces lightweight throttling to control bursts of Vertex AI requests.

Google Cloud Architecture

The application is containerized and deployed on Google Cloud Run.

Vertex AI provides:

  • Gemini for investigation and resolution
  • Gemma MaaS for business severity
  • text-embedding-005 for historical memory

Firestore provides:

  • workflow_active for current operational state
  • audit_events for historical decision events

The result is a lightweight cloud architecture where the deterministic workload stays efficient while AI is reserved for the difficult exceptions.

Challenges we ran into

The hardest part was not getting an LLM to classify a transaction.

The difficult part was designing an agentic system that could be useful in a financial workflow without turning model output into uncontrolled financial action.

Where should AI be used?

Sending every transaction to an LLM would be slower, more expensive, and unnecessary.

The architecture, therefore became

Rules for the obvious cases. AI for the exceptions. Humans for uncertainty.

That separation became the foundation of the architecture.

How do we control concurrent AI requests?

The remaining breaks are naturally parallel, but unrestricted concurrency can create bursts of requests against Vertex AI.

A bounded worker pool and throttling were introduced to process multiple exceptions concurrently while controlling request pressure.

How do we make RAG trustworthy?

A high similarity score does not automatically mean that historical information is valid business policy.

Breakwater therefore distinguishes between near-duplicates and historical precedent while filtering historical context to previously resolved cases.

How do we handle model uncertainty?

A convincing model response can still be wrong.

The Resolver therefore applies explicit business policy on top of the AI output instead of allowing the model to directly authorize autonomous resolution.

How do we preserve the decision history?

Financial operations need to know not only what the current answer is, but also how that answer was reached.

That led to the separation between:

Current workflow state

and

Historical decision events

Accomplishments that we're proud of

The biggest accomplishment is that Breakwater is not simply an LLM wrapped around a CSV upload.

It demonstrates a complete exception-management loop:

Ingest → Match → Investigate → Retrieve Precedent → Assess Risk → Resolve Safely → Escalate → Human Approval → Audit → Export

Specialized agent responsibilities

  • Matcher → reconcile records
  • RAG Memory → retrieve historical precedent
  • Investigator → explain the break
  • Severity Tagger → assess business risk
  • Resolver → apply the resolution policy
  • Human Operator → handle uncertainty
  • Audit Layer → preserve decision history

Multiple Google AI models

Different Google AI models are used for different jobs rather than forcing one model to do everything:

  • Gemini for investigation and resolution
  • Gemma for business severity
  • text-embedding-005 for semantic historical memory

Human control is built in

HITL is part of the core workflow.

An ambiguous case can move from AI investigation to human review and back into the workflow without bypassing the audit path.

Working cloud deployment

The application is containerized for Google Cloud Run, with Vertex AI providing the intelligence layer and Firestore providing operational and audit persistence.

A real operational interface

Breakwater includes a working dashboard for:

  • uploading transaction feeds
  • monitoring reconciliation
  • inspecting individual breaks
  • viewing agent results
  • reviewing escalated cases
  • approving resolutions
  • exporting reconciliation results

The project demonstrates the complete journey from raw financial feeds to an auditable resolution.

What we learned

The biggest lesson from building Breakwater was that agentic architecture is less about adding more AI and more about defining where AI should and should not operate.

The strongest pattern I found was

Deterministic where possible.
Agentic where reasoning is needed.
Policy where autonomy is risky.
Human where uncertainty remains.
Audit everywhere.

I also learned that specialized agents are easier to reason about and govern than one large "do everything" prompt.

Each Breakwater stage has a narrow responsibility, which makes the system easier to understand, debug, and govern.

Another important lesson was around memory.

A traditional knowledge base can tell an agent what something means.

Historical resolution data can tell an agent how similar problems were handled before.

That distinction makes RAG much more valuable in operational workflows.

Most importantly, I learned that the real engineering challenge in enterprise AI is often the control plane around the model:

  • What can the model do?
  • Under what conditions can it act?
  • What happens when confidence is low?
  • What information can it trust?
  • When must a human intervene?
  • How is the decision preserved?

Those questions shaped Breakwater as much as the model selection did.

What's next for Breakwater

The current prototype focuses on ledger-to-processor reconciliation, but the underlying pattern is much broader.

Broader Enterprise Use Cases

The same architecture could extend beyond ledger-to-processor reconciliation.

Potential applications include:

  • merchant settlement reconciliation
  • treasury and bank statement reconciliation
  • broker and custodian settlement
  • ERP versus subledger reconciliation
  • cross-border remittance
  • invoice and payment exception management

Durable Asynchronous Processing

The current implementation uses an in-process background worker pool.

The next major step is to move this workload to a durable queue and worker architecture so long-running reconciliation jobs can survive instance restarts and scale independently from the API layer.

Stronger Organizational Memory

I want organizations to be able to explicitly curate approved resolution patterns as business knowledge rather than relying only on historical workflow events.

This would allow institutional knowledge to become part of the reconciliation workflow.

Configurable Resolution Policies

Different organizations have different risk tolerances.

Future versions could support configurable:

  • confidence thresholds
  • break types
  • severity requirements
  • approval requirements
  • escalation policies

The longer-term vision is not simply to automate reconciliation.

It is to build a general-purpose agentic exception-management platform for enterprise operations — one that can process large volumes of routine work, isolate the difficult cases, resolve what is safe, and know when to ask a human for help.

Breakwater is controlled autonomy for financial operations.

Built With

  • chart.js
  • fastapi
  • google-adk
  • google-cloud-firestore
  • google-cloud-run
  • google-gemini
  • google-gemma4
  • google-genai
  • html5
  • python
  • vanilla
Share this project:

Updates

Submission history