War Room

ao usage

Inspiration

Treasury teams don't fail because they lack dashboards. They fail because when the cash position moves, nobody can answer why fast enough. The forecast changed — was it a customer slipping, a payment processor declining more cards, an FX obligation, or a stale assumption from three weeks ago? Answering that today means a treasury analyst pulling five systems into a spreadsheet for two days, by which point the decision window has closed.

We wanted to see whether a team of agents could compress that: not a chatbot that summarizes a ledger, but something that behaves like a war room — investigates in parallel, argues with itself, gets stress-tested, and hands the CFO a plan with every number traceable to a source.

The constraint we set ourselves up front: an LLM never does arithmetic. In a financial system, a hallucinated covenant ratio isn't a bug, it's a liability.

What it does

WAR ROOM runs a weekly 13-week rolling direct-method cash forecast for a fictional company (NovaTech, ~$450M revenue, multi-entity, multi-currency). When the weekly cycle detects a treasury-policy breach, it escalates into an investigation:

Breach detection — the cycle checks the forecast against a configurable TreasuryPolicy (min unrestricted cash, min 30-day liquidity, max revolver utilization).
Commander decomposition — the orchestrator picks which specialists are relevant to this breach rather than fanning out to all of them.
Parallel investigation — six specialists run concurrently: Forecast, Variance, AR Collections, AP Optimization, Supplier Risk, and Dodo Revenue. Each returns a structured finding with evidence references and a confidence score.
Conflict resolution — Supplier Risk can and does reject AP's proposed deferrals. The Commander resolves through follow-up investigation, not by picking whoever answered first.
Scenario generation → constraint check → stress test — candidate strategies are scored, filtered against hard constraints, then run through downside scenarios (AR −10/−20%, revenue decline, combined stress).
Replan on failure — the golden-path demo is built so the first strategy fails stress testing and the Commander replans. That failure is the point of the demo.
Approval routing — every action lands in a delegation-of-authority matrix (config/doa_matrix.yaml) that routes by action class and amount band, enforcing that preparer, reviewer, and approver are three different people.

The UI is four screens — Forecast, War Room, Recommendation, Evidence Explorer — and every figure on them is clickable back to the agent finding, source data, calculation, and assumption that produced it.

How we built it

Backend — Python + FastAPI, split along a hard line:

backend/finance/ is deterministic and LLM-free: Money, FX, cadence, forecast engine, variance bridge, covenants, controls, audit, execution. mypy --strict runs on this package, and a scripts/check_no_float.py gate fails CI if a float ever touches money. Everything is Decimal or integer minor units with explicit currency.
backend/agents/ + backend/orchestrator/ hold the reasoning: six specialists, the weekly cycle, the Commander path, conflict detection, stress and replan. Agents call the engine through a tool protocol (backend/tools/) with two implementations — EngineToolset for real runs, FixtureToolset for deterministic tests.
backend/contracts/ — shared Pydantic contracts so agents and engine speak one schema.

Model routing lives in config, not code. config/models.yaml sets per-role provider/model, temperature 0, and — the part we're most attached to — hard token budgets per role (system prompt 800, context pack 600, tool results 1500, total input 4000). CI asserts them, so a prompt edit that doubles context fails the build instead of quietly tripling cost. The executor has bounded concurrency and rate limiting, because nine concurrent agents on a free tier means "queued," and queued is an honest status.

Dodo Payments is a real financial input, not a checkout button: backend/integrations/dodo/ handles the client, Standard Webhooks verification, payout lag modeling, decline taxonomy, and success-rate metrics. Declining payment success flows into at-risk collections, which flows into the forecast, which is what trips the breach.

Schema Cartographer (backend/ingest/) — introspect an unknown source schema, match against templates, reconcile, freeze the mapping, and detect drift afterward. Real treasury data never arrives in your schema.

Frontend — Next.js + TypeScript + Tailwind, with an EvidenceProvider and a Traced primitive that makes provenance a component rather than a feature bolted on later.

64 test files across finance, seed, tools, agents, ingest, integrations, and end-to-end. make demo resets a DB, seeds NovaTech at SEED=42, applies the shock pack, and asserts the full narrative against golden fixtures.

Challenges we ran into

Keeping the LLM out of the math without making the agents useless. The easy version is to let the model compute; the correct version is to give it tools and let it decide what to investigate. Getting the boundary right — agents for hypothesis, routing, interpretation, and trade-offs; code for every number — took several passes through the tool protocol.

Making a multi-agent system deterministic enough to demo. Six agents, parallel execution, a conflict, a stress failure, and a replan — and it has to produce the same narrative every time. We solved it with a FakeProvider plus FixtureToolset so the whole war-room path runs offline in CI, with a golden-fixture assertion on the narrative.

Conflict resolution that isn't theater. It's trivial to script "Supplier Risk disagrees." It's much harder to make the disagreement load-bearing — where Supplier Risk's rejection actually removes cash from the strategy, which actually changes whether it survives stress testing. That coupling is what made the replan real rather than staged.

Money. Every Decimal-vs-float slip, every implicit currency, every rounding site. The no-float gate caught things we'd otherwise have shipped.

Model IDs and terms. config/models.yaml still carries a flag in its own comments that the model IDs need verifying against a live models-list call, and that free-tier retention/training terms are a procurement blocker before any tenant ledger data goes to an API. We'd rather leave that flagged in the config than pretend it's resolved.

Accomplishments that we're proud of

Provenance all the way down. Any number on any screen opens to agent finding → source → calculation → assumption → confidence. Nothing is unexplainable.
The no-float gate and strict-typed finance core. A financial system where the arithmetic layer is mechanically prevented from drifting.
A demo whose first plan fails. Most agent demos show a clean success. Ours shows a strategy dying in stress test and the Commander replanning — because that's the behavior that actually distinguishes an agentic system from a template.
Delegation of authority as data. The DoA matrix is a YAML file with amount bands, and route_approval rejects an amount matching two bands — an editing mistake fails loudly at routing time rather than quietly approving at the lower level.
Token budgets enforced in CI. Cost discipline as a build gate.
Dodo as a genuine treasury signal — declines and payout lag propagate into liquidity decisions.

What we learned

Deterministic code and agents aren't competing approaches; the quality of the system is set by exactly where you draw the line between them. Every time we moved that line toward the LLM, reliability dropped and the output got harder to defend.

Structured outputs with explicit confidence and failure states changed the character of the agents. Once an agent could say "I need more information" or "I reject that recommendation," the orchestrator had something real to orchestrate.

Graceful degradation had to be designed, not retrofitted. "Dodo unavailable → use last-known forecast at reduced confidence → flag recommendation for human review" is a better product behavior than either crashing or silently guessing.

And the hardest part of a CFO tool isn't the analysis. It's making the analysis auditable.

What's next for War Room

Verify the model routing — run the live models-list call, pin real IDs, context windows, and function-calling support, and settle the data-retention terms before any real ledger data leaves the box.
Broaden the specialist roster — the architecture has slots for dedicated Debt, Covenant, and FX agents; today those live mostly in the deterministic engine.
Real execution paths behind approvals — currently the approval routes to a decision; next is executing the approved action through the integration layer, with full reversal handling.
Forecast accuracy learning loop — we already track accuracy and variance week over week; the next step is feeding realized error back into agent confidence calibration.
Postgres + multi-tenant — the model layer is SQLModel with Alembic migrations, currently demoed on SQLite.
Continuous monitoring rather than a weekly cycle — event-driven breach detection so the war room opens the hour the shock lands, not the following Monday.