ClosePilot

Meet ClosePilot An autonomous finance agent built to take month-end reconciliation from messy records to a close-ready result.
From bank transactions to reconciled records. ClosePilot matches, investigates, and flags what needs a human.
Every decision needs evidence. ClosePilot connects transactions, ledger records, documents, and audit evidence.
When the evidence isn't enough, ClosePilot doesn't guess. It investigates, explains the risk, and puts the decision back in human hands.
The finish line: an evidence-backed close package with reconciliation results, exceptions, decisions, and full traceability.

Inspiration

Month-end reconciliation is a repetitive but important finance workflow. Teams have to compare bank transactions with ledger records, investigate discrepancies, find supporting evidence, and decide which cases are safe to resolve.

We wanted to build something beyond an AI chatbot — an agent that could actually execute this workflow.

That idea became ClosePilot: an autonomous finance operations agent designed to automate high-confidence reconciliation while keeping humans in control when the evidence isn't sufficient.

What it does

ClosePilot automates the month-end reconciliation workflow from end to end.

It:

Validates and normalizes financial records
Matches bank transactions with ledger entries
Detects discrepancies and exceptions
Investigates exceptions using available evidence
Automatically resolves high-confidence cases
Escalates uncertain cases to human review
Maintains evidence behind decisions
Produces a reconciliation result and close-ready package

The core principle is:

Automate high-confidence finance work. Escalate uncertainty to humans.

ClosePilot isn't designed as a generic CFO chatbot. It actually performs a finance operations workflow.

How we built it

We built ClosePilot using a combination of deterministic financial logic, agent orchestration, evidence handling, human review, and automated evaluation.

Deterministic tools handle operations such as validation, vendor normalization, transaction matching, duplicate detection, and financial calculations. The agent orchestrates these tools, investigates exceptions, gathers evidence, and determines when human review is necessary.

We also created a synthetic financial dataset with ground truth so we could evaluate the system objectively.

Our development loop was:

Build → Evaluate → Analyze failures → Improve → Evaluate again

AO was used during the build to help develop and iterate on the system, and its sessions are included in our development story.

Challenges we ran into

One of our biggest challenges was making sure ClosePilot wasn't simply tuned to our demo dataset.

During development, we found transaction-specific logic that could make individual fixture cases appear correct without representing a general reconciliation strategy.

We removed those hardcoded behaviors and made the system reason from the actual transaction, ledger candidates, reconciliation results, calculations, and evidence.

This exposed genuine ambiguous cases in our evaluation data.

Rather than modifying the fixture or ground truth to force a perfect score, we kept them unchanged and treated those cases as real failures to investigate.

We also implemented an evidence-first human review state machine, preventing approval when required evidence hasn't been supplied.

Accomplishments that we're proud of

We're proud that ClosePilot became more than a demo — it became a testable finance workflow.

Our latest verified build has:

207/207 automated tests passing
0 TypeScript typecheck errors
Successful production build
100% exception recall
0% false auto-close rate
75% overall agreement with our 8-case ground-truth evaluation

We're especially proud of the 0% false auto-close rate. For a finance workflow, safely refusing to make an uncertain decision can be more valuable than maximizing automation.

We also built human review directly into the workflow instead of treating it as an afterthought.

What we learned

We learned that building an autonomous agent isn't just about giving an LLM more capabilities.

The surrounding system matters just as much.

We learned to:

Keep deterministic financial operations deterministic
Require evidence behind important decisions
Build human oversight directly into the workflow
Use ground-truth evaluation instead of relying on a successful demo
Analyze failures instead of hiding them
Avoid overfitting the system to a specific dataset
Treat uncertainty as something the agent should recognize, not something it should blindly overcome

Most importantly, we learned that a good finance agent needs to know when not to act.

What's next for ClosePilot

The next step is improving how ClosePilot handles genuinely ambiguous financial records without introducing fixture-specific rules.

We want to improve the reconciliation engine, expand the evaluation dataset, strengthen exception investigation, and make the system increasingly reliable across different month-end scenarios.

Our long-term vision is for ClosePilot to become a broader autonomous finance-operations layer for the CFO's office — starting with reconciliation and expanding into other repetitive, evidence-driven close workflows.