Inspiration

Robotics installers recognise revenue as work progresses, so every forecast change moves the books. The rule is cost-to-cost:

$$\text{revenue} = \text{contract value} \times \frac{\text{costs incurred}}{\text{total estimated costs}}$$

That makes a casual site note dangerous. "We need another $200,000 to finish" can mean $200,000 of additional cost, or a new remaining total of $200,000. On a $1,000,000 contract, those two readings are roughly a quarter of a million dollars of revenue apart. Someone has to decide what the note meant, recompute, and book the adjustment, and today that is usually a spreadsheet and a guess.

This is not hypothetical. Symbotic, a public warehouse-automation company, disclosed a material weakness in 2026 over exactly this kind of revenue timing. That shows the problem is real at a major robotics company. It does not show Symbotic would buy Robaudit, and we don't claim it does.

We wanted an agent that does the tedious part (read, reconcile, recompute, draft the entry) but asks a human the moment a word is ambiguous, and never touches the ledger on its own.

What it does

Robaudit is an accounting platform for robotics installers. Summo is its agent and mascot: one coordinator that reads every new document, decides whether to ask a question, recalculates, and proposes the journal entry. Summo runs a loop of specialist steps: two readers (Claude and Gemini), a document reader (Azure Document Intelligence), a reconciler, a phone channel (Convex plus Cartesia), an arithmetic engine, an explainer (Claude), a proposal step, and a posting guard. It is one coordinator with specialist steps, not nine separate agents.

One human, the controller (Dana in our demo), is the only role that can approve or post.

The loop, step by step

  1. A source arrives. The deployment manager types or speaks a site note on their phone, or photographs a forecast sheet. It lands in the SpacetimeDB ledger as a source row.
  2. Two readers extract facts. Claude and Gemini each pull facts independently, with source spans: the quote, the amount, and an ambiguity flag. Photos go through Azure Document Intelligence first.
  3. Reconcile. If the readers disagree on the amount, or either flags ambiguity, Summo asks instead of guessing. If a saved convention already answers the question, Summo cites it and skips the call.
  4. The call. Convex opens a clarification. Cartesia Sonic speaks the question on the manager's phone, Cartesia Ink transcribes the spoken answer, and the manager confirms the transcript. The answer is stored as evidence, not chat.
  5. Recalculate. A plain TypeScript engine in integer cents recomputes progress, revenue, contract asset, margin and the adjustment. The model never does arithmetic.
  6. Propose. Summo writes the entry, a plain-English explanation, and the evidence ids. A loss warning is raised when margin hits zero.
  7. The controller decides. Dana approves and posts. Approving a proposal that carries a pending convention saves that convention.
  8. It improves. Next month, "another $50,000 for conveyor rework" posts with no call, citing the saved convention.

The numbers (Project OAK, $1,000,000 contract)

Before → after the manager confirms "additional $200,000" (in brackets: what a "new total" reading would have given):

  • Remaining cost: $400,000 → $600,000 ($200,000)
  • Progress: 50% → 40% (66.7%)
  • Cumulative revenue: $500,000 → $400,000 ($666,667)
  • Contract asset: $200,000 → $100,000
  • Expected margin: $200,000 → $0, with a loss warning ($400,000)
  • Adjustment: Dr Revenue 100,000 / Cr Contract asset 100,000 (+$166,667)

Every figure matches an independent answer key, and the demo script asserts them against the live ledger.

What is real and what is simulated

  • Real: the SpacetimeDB ledger on Maincloud, Convex production, Summo on Azure Container Apps, Claude Opus 5.5, Gemini 3.5 Flash Lite, Cartesia Sonic and Ink, Azure Document Intelligence.
  • Simulated: Aster Robotics, the Oakland project and its data, the six past closes behind the seeded convention, and the controller identity.
  • You: the judge plays the deployment manager, on their own phone.

How we built it

Each layer has one job, and only the ledger is the system of record.

  • Ledger: SpacetimeDB on Maincloud. System of record with fifteen reducers. Controls live in reducers, not the UI: role, revision, staleness, open period, debits equal credits, unique posting reference.
  • Agent: Summo on Azure Container Apps. Watches the ledger and runs the reader, reconcile, engine, explain and propose steps. Has no permission to post.
  • Readers: Claude Opus 5.5 and Gemini 3.5 Flash Lite. Independent fact extraction with source spans. Claude also writes the plain-English explanation.
  • Documents: Azure Document Intelligence. Reads photographed forecast sheets into text the readers can extract from.
  • Human channel: Convex (production). Live queries drive the phone and call pane; mutations move a call through ringing, answering and confirmed; actions call Cartesia; file storage holds question and answer audio; the action-retrier component wraps both voice actions; anonymous auth protects answer mutations.
  • Voice: Cartesia Sonic 3.6 and Ink 2. Sonic speaks the question, Ink transcribes the answer, both inside Convex actions so the key never reaches the browser.
  • Engine: plain TypeScript, integer cents. Pure functions, no I/O: progress, revenue, contract asset, margin, adjustment.
  • Audit mirror: Atlassian Forge app plus a ledger bridge. Mirrors every ask, proposal, return and post into Jira project SUMMO with its reasoning. Jira never writes to the ledger.
  • Web app: Azure Static Web Apps. Landing page, the judge's phone page, and the controller's workbench.
  • Design: Figma. Design file pages 00 to 08, a handoff page with every screen captured live, and a Slides deck.

The workbench covers Inbox, Agents, Projects, project page, Close, Evidence, Audit trail, Policies, Connections and Team.

Challenges we ran into

  • Pivoting past the halfway mark. We switched to Robaudit more than halfway through the hackathon, so we scoped hard: one project, one contract type, one controller.
  • Deciding when to ask. Asking on every note is annoying; never asking is dangerous. We settled on a rule (ask when the readers disagree on the amount or either flags ambiguity, unless an approved convention answers it) and tested it against twenty freeform phrasings.
  • Keeping the model away from the money. Models are good at reading and bad at arithmetic and authority. We split the work so the model only reads, asks and explains, while a pure integer-cent engine does the math and ledger reducers decide what can post.
  • Making voice reliable enough for a live demo. A phone call in a loud hackathon hall can fail in many ways. We handled mic failure, upload failure, a declined call and misheard answers, and kept a confirm-the-transcript step so a bad transcription never becomes evidence.
  • Putting controls in the right place. A UI check can be bypassed. Moving role, staleness, period and balance checks into SpacetimeDB reducers meant the "Break it" buttons hit the real rules, not a front-end guard.
  • Customer understanding. Our evidence that installers need this rests on one SEC filing. We have not interviewed controllers yet.

Accomplishments that we're proud of

  • A judge can join from their own phone, answer a real voice call, and watch revenue move from $500,000 to $400,000 within a second of confirming.
  • Every figure in the demo matches an independent answer key to the dollar, asserted against the live ledger.
  • Sixteen planted errors, each pinned by a test, including a model inventing a figure, an agent trying to post, and a prompt-injection note.
  • The agent cannot post, and that is enforced in the database, not the interface.
  • Month three posts with no call, citing a convention a human approved: learning that a person can audit, not hidden in weights.
  • A full reset between judges takes under a second.

What we learned

  • Why a bigger cost estimate lowers revenue. Progress is costs incurred over total estimated cost, so a bigger total means a smaller share is done. Nothing was a mistake; an estimate changed, and cash is untouched.
  • Agents earn trust by asking. The most valuable thing Summo does is notice that a sentence has two meanings and route it to the one person who knows.
  • Two readers beat one. A second, independent model is a cheap check. Disagreement on an amount is a strong signal to stop and ask.
  • Evidence beats chat. Storing the manager's confirmed voice answer as a source on the entry turns a conversation into something an auditor can replay.

What's next for Robaudit

  • Talk to controllers. Interview finance teams at robotics and automation installers to test whether this is the pain we think it is.
  • Real integrations. Pull cost exports and forecasts directly from the ERP and project tools installers already use, instead of seeded data.
  • Real identity. Replace the simulated controller with proper sign-in and role assignment.
  • Robotics-as-a-service contracts. RaaS subscriptions recognise revenue over the contract term, not cost-to-cost, so supporting them means a second engine alongside the current one.
  • Longer voice clarifications. Move from one short answer to a short back-and-forth when the first answer is still unclear.
  • Broader policy scope. Today we cover a single performance obligation, fixed price, cost-to-cost. Harder ASC 606 cases are out of scope by policy until a human-reviewed path exists for them.

Built With

  • anthropic
  • atlassian-forge
  • azure
  • azure-container-apps
  • azure-document-intelligence
  • azure-static-web-apps
  • cartesia
  • claude
  • convex
  • figma
  • gemini
  • jira
  • spacetimedb
  • typescript
Share this project:

Updates

Submission history