Inspiration
People aren't good at their work because they never make mistakes. They're good because they notice them, undo them, and don't make the same one twice. A person is a long chain of corrected mistakes, and a finance team is built on that: a controller's correction becomes the team's rule, a loss becomes a policy, a new hire earns trust one check at a time.
Agent systems don't work this way. An agent is frozen the day it ships: it repeats its mistakes, or it learns whatever it is told, a person's mistakes included. We wanted an AI finance team that corrects itself the way people do. It catches its own mistakes, learns from every correction, refuses the wrong lessons, and carries its experience to the next company, while a kernel it cannot edit keeps the money safe.
What it does
Charter runs the money desks of a finance team: cash application, collections, payables, payments, refunds, a mailroom and an internal auditor. These seven seats act only through a kernel.
- It catches its own mistakes. When a later payment contradicts an earlier inference, it shows the controller the evidence and, on a yes, undoes it.
- It learns from corrections. A model turns a person's answer into a candidate rule, a replay gate runs that rule against every past decision, and a person switches it on.
- It learns from losses. After a debit bounced, an exhaustive replay of every past payment found the one rule that would have stopped it. That rule now guards every payment, even one approved by both controllers.
- It refuses the wrong lessons. A correction that would break a contract is declined and never learned. A verdict that contradicts a rule with a clean record goes back to the person for a second look.
- It checks the team. A solver found what no single agent owns: collections chasing a customer whose own money was waiting to be applied. Two lines of constitution now forbid it, proved for every situation.
- It takes its experience along. A sister company that started from these rules made 2 mistakes in its first three months instead of 20.
Results
- On a locked test split it never trained on, 310 of 314 payments against real invoices (UCI Online Retail II) landed where the payer meant.
- On that same split, the day-one cash skill made 18 mistakes acting alone. After four learned versions it makes 2.
- 36 attack emails: none got through, while tool-using LLM agents fell for 13, 8 and 4.
- On a real bank's reconciliation data (BenchRec), its learned matching rules matched 81.5% of lines at 99.95% precision.
- $89,242 of duplicate bills stopped, none paid.
How we built it
Everything lives in SpacetimeDB. The kernel and the seats are one TypeScript module. Every action, by every agent and every person, is a reducer call: the kernel checks it against permissions and a constitution of 18 articles, then appends it to a hash chain.
- Models read. Two model families (Qwen and Kimi on Alibaba Cloud Model Studio) read every email, hold no tools, and file what they read as untrusted facts. If they disagree, a person decides.
- Programs decide. Every decision is a small rule program in our own DSL. One deterministic interpreter runs it in the database and in our tools, and a line-for-line port runs it in the browser. Cash matching is an exact subset-sum search (meet-in-the-middle) that counts its solutions and records which matches are inferences.
- Learning is gated. The evolver writes candidate versions from people's answers, and the replay gate runs each one over every past decision before a person can switch it on. A new mistake on a case people have checked is tolerated only if \(\mathrm{UB}_{95}(k,n) \le \tau\), the exact binomial upper bound against the desk's tolerable error rate. One mistake is priced at \(1/\tau\) questions. Each rule then earns its autonomy: spot checks start at 25% and fall to 3%.
- The team is checked as a team. A Z3 encoding of the same TypeScript checks the seats and the kernel together.
The live UI is React on SpacetimeDB subscriptions. The public page replays the runs in the browser on a canvas network. The data comprises:
- real invoices from UCI Online Retail II for two companies;
- a simulated franchisor with planted errors;
- BenchRec bank data.
Answer keys are hidden from the system, and every model call is cached, so any run replays exactly.
Challenges we ran into
- Our own evaluator kept catching us. The evolver overfit a single example until we chose candidates by robustness. Our first statistical gate let through a candidate that was wrong on 51 of its 52 new decisions.
- Messy payments broke learning. Once bank charges were in the data, "apply to the invoice the payer named" and "apply oldest first" looked like contradictory answers. Decisions now record where each outcome would put the money.
- People slip. One planted slip in a spot check spread to 182 near-identical cases. A verdict now counts only after the person stands by it on a second look.
- Right alone, wrong together. No test found the collections bug; a solver did.
- Iterating against your own gate overfits it. When we let the model retry against the gate's counterexamples, the extra lessons it got through were worse by the answer key. That is why a person still approves every version.
What we learned
- Correcting mistakes is a property of the system, not of the model. The model will be wrong; the system has to notice, undo and learn without putting money at risk.
- A learner has to be able to refuse a lesson.
- Agents that are each right can be wrong together, so check the team, not just the agents.
- Measure only on what the system never trained on, and keep everything replayable.
Log in or sign up for Devpost to join the conversation.