Ledger Forge

Ledger Forge
How the Ledger Forge Works

Inspiration

Bank-to-general-ledger reconciliation is still a highly manual process for finance teams. Accountants spend hours matching bank transactions against ledger entries, investigating amount differences, identifying duplicates, resolving timing differences, and determining whether an unusual transaction is actually an error. The real challenge isn't finding a match — it's knowing when there is enough evidence to safely act.

We built LedgerForge to transform this repetitive, spreadsheet-driven workflow into an agentic, evidence-first reconciliation system for the Office of the CFO — automating high-confidence work while ensuring ambiguous or risky financial decisions are escalated to a human instead of being guessed by an LLM.

What it does

LedgerForge automates bank-to-GL reconciliation end-to-end while keeping financial authority deterministic and the human firmly in control:

Transaction Ingestion & Normalization: Ingests bank and general-ledger transactions and normalizes amounts, currencies, dates, references, transaction direction, and counterparty information.
Deterministic Matching: Generates and ranks candidate matches using references, dates, normalized economic amounts, currency, direction, and multi-field similarity — without relying on an LLM to perform the core financial matching.
Ambiguity Detection & LLM Reasoning: Routes difficult cases such as multiple candidates, partial payments, amount variances, duplicate candidates, unusual timing, and fuzzy references to an LLM specialist for structured diagnostic reasoning.
Fail-Closed Decision Engine: A deterministic Decision Engine remains the supreme authority and blocks automatic reconciliation for currency conflicts, direction conflicts, duplicate consumption, material variances, high-risk anomalies, conflicting precedent, or insufficient evidence.
Accountant Exception Review: Escalated transactions enter a structured review workspace where accountants can inspect evidence, AI recommendations, candidate matches, historical precedent, and the reason the system stopped before recording a resolution.
Reconciliation Memory: Human resolutions become structured historical precedents that can be retrieved for future exceptions. Conflicting precedents are surfaced rather than arbitrarily selected.
Controlled Agent Self-Improvement: An offline optimization loop analyzes failures and human feedback, proposes candidate policy improvements, evaluates them against regression data and zero-tolerance safety gates, and requires explicit human promotion before activation.
CFO Policy Governance: Provides visibility into the active agent version, decision policy, safety boundaries, validation status, and benchmark results so finance teams can understand what the agent is allowed to do and why.

How we built it

We architected LedgerForge as a layered agentic finance stack prioritizing deterministic decision-making, safety, auditability, and human governance:

Agentic Workflow (AO Agents & Python): Built an agentic workflow that separates transaction matching, LLM investigation, deterministic decisioning, accountant review, memory, evaluation, and policy optimization into distinct stages.
Deterministic Matching & Decision Engine: Implemented financial matching and safety checks using structured rules for amounts, currencies, references, dates, direction, duplicates, and material variance.
AI & LLM Integration: Integrated controlled LLM reasoning for ambiguous reconciliation exceptions. The LLM provides recommendations and diagnostic evidence but has no final decision authority.
Reconciliation Memory: Built structured SQL-based memory that retrieves historical accountant resolutions using deterministic similarity signals rather than relying on opaque vector retrieval.
Agent Evaluation & Optimization: Built an evaluation framework that benchmarks candidate agent policies against regression data and enforces zero-tolerance safety gates before any candidate can be promoted.
Human-in-the-Loop Governance: Structured accountant resolutions into typed feedback categories so human decisions become usable learning signals while preserving the original machine decision.
Backend & Data Layer: Built the backend with Python, FastAPI, Pydantic, SQLite, and Supabase/PostgreSQL support for reconciliation data, feedback, agent versions, and audit records.
Frontend: Built a React-based interface for reconciliation results, exception review, audit history, historical precedent, and CFO policy visibility.
Auditability: Added additive audit trails covering machine decisions, LLM recommendations, memory retrieval, human resolutions, and controlled agent-version promotion.

Challenges we ran into

LLM Authority Boundaries: Designing an agentic system where the LLM can investigate ambiguous financial cases without ever being able to override deterministic safety rules or make unauthorized financial decisions.
Financial Exception Complexity: Distinguishing legitimate reconciliation scenarios such as partial payments, timing differences, bank fees, duplicates, missing entries, and amount variances instead of treating every mismatch as an error.
Safe Self-Improvement: Connecting human feedback to future agent improvements while preventing an agent from silently changing its production policy. Candidate versions must pass regression and zero-tolerance safety checks and require explicit human promotion.
Currency & Transaction Safety: Preventing cross-currency comparisons, direction conflicts, duplicate consumption, and material amount variances from becoming automatic matches.
Preserving Audit History: Ensuring an accountant correction does not overwrite the original AI recommendation. Machine decisions, human resolutions, and subsequent agent improvements remain separately traceable.

Accomplishments that we're proud of

105/105 Backend Tests Passing: Verified the complete backend workflow across policy wiring, structured feedback, reconciliation memory, LLM reasoning, agent optimization, and accountant exception review.
Zero-Tolerance Safety Gates: Candidate agent versions are rejected when they introduce false automatic matches, currency confusion, duplicate auto-matches, direction conflicts, material-variance auto-matches, high-risk auto-matches, or regression failures.
Model as Specialist, Never Judge: Successfully separated LLM reasoning from financial authority — the model investigates and recommends while deterministic policy and safety controls make the final decision.
Human Feedback as a Learning Signal: Accountant resolutions are converted into structured feedback and can influence future agent policy proposals rather than remaining as unstructured notes.
Controlled Self-Improvement: LedgerForge can analyze failures, propose new policies, benchmark them, and prepare candidates for promotion without allowing autonomous production mutation.
Explainable Exception Review: Accountants can see the evidence, candidate matches, AI reasoning, historical precedent, and exact reason LedgerForge stopped before making a decision.

What we learned

In high-stakes financial workflows, agentic does not mean autonomous at all costs. The most useful financial agent is one that can investigate independently, automate high-confidence work, learn from human decisions, and reliably know when it does not have enough evidence to act.

What's next for LedgerForge

Banking & ERP Integrations: Connect LedgerForge with real banking feeds and accounting/ERP systems while preserving its deterministic safety boundaries.
Larger-Scale Reconciliation: Extend the system to handle higher-volume daily and monthly reconciliation workloads.
More CFO Workflows: Expand the agentic architecture into additional Office of the CFO workflows such as variance investigation, close management, and financial exception analysis.
CFO Risk Analytics: Add higher-level visibility into recurring exceptions, unresolved reconciliation risk, automation rates, and policy performance.