Settle

• 

  End-to-end verifier: nine independent checks re-derive the whole run from the database.

• 

  Prompt injection from a verified customer domain: the agent ignores it and Settle flags it for the team.

• 

  A dispute judged on evidence. Claude checks the carrier's delivery record and proposes a pro-rata credit, policy routes it for approval.

• 

  Memory ON vs OFF ablation scored against hidden ground truth, plus a reproducibility proof that gives an identical audit head on re-run.

• 

  Command center: a simulated week of collections runs autonomously, and every line links to the decision and reasoning behind it.

• 

  Headline results: human touches 5 vs 15 with memory, 97.8% auto-resolved, zero unsafe actions, and 9/9 verification checks passing.

• 

  Settle: the accounts-receivable agent that learns how customers pay, can't be talked into moving money, and proves every action.

Inspiration

Every B2B company waits for its money. Accounts receivable means collectors chasing invoices across billing, CRM, email and the bank, and every exception (a dispute, a short payment, an unknown payer) needs a person. It's a natural fit for an AI agent, but it's also where an agent is most dangerous: it moves money, it reads email from strangers, and finance teams have to be able to prove what happened to an auditor.

I wanted to build an agent a controller would actually let run, because it gets better on its own, it can't be talked into a bad payment, and every action it takes can be checked.

What it does

Settle runs collections on its own:

• Works the books. It sends reminders, works out why an invoice is late, judges disputes on evidence (for example, checking the carrier's delivery record before proposing a pro-rata credit), applies incoming bank cash to the right invoices, and books it to the ledger.
• Learns. Its memory is episodic, semantic and procedural, and only human verdicts and observed payments are written to it, never the agent's own guesses. Repeated outcomes become playbooks ("this customer always pays ~3 weeks late"). Once a controller approves a playbook, it runs before the LLM, so the next case costs nothing. Drift detection suspends a playbook when a customer's behavior changes.
• Can't be talked into moving money. Guardrails are code that runs after the agent decides; they aren't prompts. Every decision gets a tier: 0 autonomous, 1 autonomous with a notification, 2 needs approval, 3 blocked. Lookalike domains, bank-detail changes and prompt injections are blocked even if the model is fooled, and a test swaps in a fully compromised agent to prove it.
• Proves every action. Every state change is a hash-chained audit event:

$$h_i = \mathrm{SHA256}\big(h_{i-1} \,|\, \mathrm{canonical}(e_i)\big)$$

Every decision also stores the exact input the model saw. An end-to-end verifier re-derives the whole run with nine independent checks (chain, provenance, policy replay, guardrails, invoice replay, ledger tie-out, messages, memory, work items). Replaying a run reproduces the same fingerprint.

• Built to pilot. It has:
  • roles with approval limits and separation of duties;
  • shadow and supervised autonomy modes, plus a kill switch;
  • a prioritized collector worklist and auditor CSV exports;
  • CSV import of your own AR aging and bank data;
  • a one-port server or a Docker image.

Results

The test bed is a simulated B2B company with 12 customers over 3 months, scored against hidden ground truth, running Claude Opus 5.5.

Fusing Moss with SQLite FTS5 and reranking with Bitdeer lifts memory-retrieval MRR from 0.70 to 0.96. At 25× scale (300 customers, 5,640 decisions), the simulation takes 9 s and verification takes 0.33 s.

How I built it

• Claude Opus 5.5 is the agent. It calls read-only tools (invoices, payments, CRM, shipping, memory search) and must finish with one strict, evidence-cited submit_decision call. Customer email is treated as untrusted content.
• Moss holds semantic episodic memory. It's fused with SQLite FTS5, which stays the source of truth if Moss is down.
• Bitdeer AI Model Studio:
  • bge-reranker-v2-m3 reranks memory hits;
  • DeepSeek-V4.1-Flash is a cheap intent router that sends routine acknowledgements past the expensive agent;
  • an instruction guard stops injected text from being cheap-routed.
• Pipeline: playbook → deterministic cash match → router → Claude → validation (hallucinated records, stale state, amounts) → policy tiers in code → executor → audit log. The cheapest path is always tried first.
• Backend: Python, FastAPI, SSE for the live activity feed, and SQLite with WAL. Money is integer cents everywhere.
• Write path and replay:
  • Only the executor can write ledger and invoice state, and every field change is audited (old → new, actor, decision).
  • Every Claude, Moss and Bitdeer call goes through a recorder, so whole runs replay offline and deterministically.
• Frontend: React, Vite, TypeScript, Tailwind and Recharts. It has a command center, approvals, a worklist, decision provenance, entity trace, the eval and verifier screens, and a guided tour.
• Testing: 159 backend tests and 90 frontend tests, plus a done gate that runs the tests, lint, the offline verifier and the build.

Challenges I ran into

• Memory made the agent worse at first. In the first live ablation, accuracy was lower with memory on. Memory had read "pays 2 days after every reminder" as "reliable late payer, stop reminding." I split customer profiles into reminder-driven payment and intrinsic lateness, and added tests for it.
• Determinism with an LLM in the loop. Getting same seed ⇒ same audit head took three things:
  • recording every external call;
  • keeping wall-clock time, run IDs and token counts out of hashed events;
  • giving every query that drives behavior an explicit ordering, including under concurrent deliberation.
• Separation of duties at the edges. Approvers can edit amounts, so authorization uses the larger of the proposed and edited amounts. Stale approvals get a "close as stale" verdict, so dismissing them doesn't write misleading memory.
• Ambiguous cash. When a payment matches more than one combination of invoices, Settle routes it to a human with the candidates instead of guessing.

Accomplishments that I'm proud of

• The guardrails hold against a deliberately compromised agent, not just a well-behaved one.
• Anyone can verify a run from one SQLite file: the nine checks re-derive everything, and a replay gives the identical fingerprint.
• The ablation is measured and honest, including the regression I found and fixed.

What I learned

• Trustworthy agents are mostly about the system around the model: evidence requirements, policy in code, narrow write paths and replayable records.
• Memory has to be written from outcomes and human verdicts, not from what the agent believes, or it reinforces its own mistakes.

What's next for Settle

• Connectors for real ERPs and banks (NetSuite, QuickBooks, Plaid) to replace CSV import.
• A shadow-mode pilot with a real AR team, then moving customers to supervised and autonomous modes as playbooks earn trust.
• Extending the same architecture to accounts payable and dispute management.

Built With

• bitdeer
• claude
• deepseek
• docker
• fastapi
• moss
• python
• react
• recharts
• sqlite
• tailwindcss
• typescript
• uv
• vite