Inspiration
Every company reconciles the same three documents before paying an invoice: the purchase order, the goods receipt, and the invoice itself. Miss a mismatch, and the most expensive failure isn't a typo. It's a duplicate invoice slipping through and getting paid twice. AP teams do this reconciliation by hand, at scale, every week. It's a real, unglamorous pain point in the Office of the CFO, and exactly the kind of workflow that's tedious enough to automate but risky enough that you can't just let an agent guess.
What it does
Ledger Sentinel runs three-way match (PO / Goods Receipt / Invoice) end to end.
- Clean invoices auto-clear. Across a 300-invoice test set, 260 matched within tolerance and posted straight to the ledger, with no human touch.
- Anomalies escalate, with evidence. The other 40 are routed to a review queue: quantity variance, price variance, missing PO, tax mismatch, line-item mismatch, vendor name drift, and suspected duplicates, each with a plain-English summary, a step-by-step reasoning trace, and a three-way document comparison with the mismatched field highlighted.
- A human decides in one click. Approve or reject, and the ledger updates immediately.
The agent is deliberately conservative. It's tuned to escalate when uncertain, because a false auto-clear (paying an invoice that shouldn't have been paid) is far more expensive than an unnecessary escalation.
How we built it
Deterministic logic (matching, tolerance rules, ledger lookups) is separated from LLM judgement (vendor name normalisation, duplicate reasoning, plain-English summaries), using TensorMux for inference, running GLM-4.7-Flash. Every escalation records which checks were deterministic and which required model judgement, so the split is visible in the UI, not just asserted in a slide.
The whole build ran on AO, using it to orchestrate the coding-agent sessions from the dataset generator through the final UI pass.
Challenges we ran into
The one worth calling out: an early version let the LLM veto a strong deterministic duplicate signal. If the model said "not a duplicate," the system trusted it, even when the ledger had already found an exact vendor/amount/date match. Testing at scale (300 invoices, repeated runs) surfaced a real case where the model was wrong and a duplicate would have been paid twice.
We fixed the escalation logic so a ledger near-match always escalates. The model can only adjust severity and write the explanation; it can never suppress the flag. After the fix: 0 false-clears across 300 invoices and three separate runs. Finding and closing that gap mattered more to us than the number itself. It's the difference between a demo that works and a design that's actually safe to trust with real payments.
What we learned
The interesting engineering problem in a workflow like this isn't the happy path. It's designing for the failure mode you're most scared of, and being honest when your first version doesn't actually protect against it.
What's next
The current version handles the case where a PO has no goods receipt yet by auto-clearing (there's no such case in the seed data to test against), a real AP team would flag that as its own two-way-match gap. That's the next thing we'd close.
Log in or sign up for Devpost to join the conversation.