Invoiceloop

• 

  The pipeline end to end: PDF → Nutrient DWS → six deterministic gates → a named human signs → export. No path crosses without a signature.

• 

  Grounding anchors every DWS value to its source page. DWS documents the behavior itself: confidence consistently reflects page grounding.

• 

  The real invoice that started it: a radio station extracted with an advertising agency as seller. Payment heading to the wrong party.

• 

  A real shipped error: the taxpayer ID read as the invoice number. Both citations passed — the extraction looked confident throughout.

• 

  Buyer and seller swapped on a live invoice. Three vision readers rejected the assignment; the deterministic gates caught the violation.

• 

  Where automation stops: ready-for-approval. Only a signed human decision — with name and rationale — reaches approved-for-export.

• 

  The seller-side twin of the buyer swap: structure gates pass while vision flags the semantic error — evidence over confidence.

• 

  Honest negative result: 10.5% zero-touch routing, but the safety qualification failed - 6 wrong payment values, 3 unscored slots. Denied.

• 

  Evidence. Review. Named approval. Deterministic, replayable DWS output with a gate the model cannot talk past. Commit 3837ae8.

Inspiration

InvoiceLoop grew out of BriefLoop, our earlier architecture reference for claim and document review. BriefLoop established the discipline we still stand on — a single-writer control plane where models only submit drafts and deterministic code owns the ledger — but its support-sufficiency stack was the part it never finished: §8.4 of its own spec marks the semantic gate "not yet delivered."

InvoiceLoop is that cell, shipped. We took the unfinished sufficiency-of-support idea into the invoice domain, where it has a hard, checkable ground truth — and where a real failure made the stakes concrete: a radio station's invoice extracted with the seller as an advertising agency, a tax ID in the invoice-number field, the payment heading to the wrong party, and the extraction looking confident the whole time.

Nutrient DWS does the document work — parsing, extraction, structured output. We built the other half: the evidence, review and approval layer that makes DWS output something a human can put their name on.

What it does

Every field DWS extracts is bound to three evidence sources side by side: the DWS value with its cited region, an independent OCR pass, and the rendered page crop the citation points to. Six deterministic gates run on every field — arithmetic identities (net + VAT = gross = due), citation binding, dual-mode agreement (DWS understand vs agentic), presence, well-formedness, and a vision cross-read. A check that cannot run is blocking, never skipped.

The deliverable is a support matrix, not a verdict: support strength, source tiers, applicability disputes, and limitations — recorded as-is. Automation stops at ready-for-approval; only a signed human decision reaches approved-for-export. Change a value after approval and the approval goes stale: the document falls back to the queue, and the old approval stays in the ledger.

How we built it

• Nutrient DWS as the extraction engine (both understand and agentic modes; every response archived for zero-cost replay — 320 stored calls, zero failures).
• Google ADK SequentialAgent (clerk → critic → approver) on Gemini 3.7 Flash via Vertex AI, executed by an IAM-private Cloud Run Job; every model call, source document and raw response lands in Cloud Storage.
• Single-writer discipline: agents emit un-ID'd drafts only. Deterministic Python assigns IDs, freezes ledgers, runs the policy gate, and records failures the agents cannot talk their way around.
• A public read-only workbench on Cloud Run — write routes return 403, because a public endpoint must never be able to forge a human adjudication.

Challenges we ran into

Confidence doesn't separate safe from unsafe. Six pre-registered evaluation rounds hunting for a signal that flags every consequential error found none — so we stopped trusting scores and bound every value to verifiable evidence instead.

Our own safety qualification failed — and we shipped the failure. The unattended arm hit 10.5% zero-touch routing, but the zero-touch payment subset contained 6 wrong values and 3 unscored slots. The gate refused promotion; the negative result ships in the repo.

Making failure honest. A cloud job that "completes" while every model call failed is a lie with an exit code of 0. We made the runner exit non-zero on any failure — an injected fault visibly exits 1.

Accomplishments that we're proud of

• A judge can replay the whole pipeline in three commands, zero API keys, zero network (vendored invoices + stored DWS responses).
• Approval staleness enforced in code: mutate an approved value and the signature is voided, with the original kept in the ledger.
• Every claim in this write-up is checkable against frozen, hash-pinned evidence files in the repo.
• The system refuses to promote its own automation when the safety bar isn't met.

What we learned

• Extraction correctness is untrustworthy; support is verifiable. Sell the verification, not the accuracy.
• Gate availability is a safety property: a check that can't run must block, or the gate becomes a rubber stamp.
• Negative results are load-bearing: publishing why the zero-touch arm was denied is more credible than any accuracy claim.

What's next for InvoiceLoop

The improvement loop is already wired: every human adjudication becomes a labeled note, mined into cohorts that propose policy changes — which must again pass the same gates before anything ships. Next: earn the zero-touch promotion through a passing safety qualification rather than an override, and extend the evidence binding to more DWS document types.

Built With

• agent-development-kit
• agents
• ai-agents
• cloud-run
• cloud-storage
• document-intelligence
• document-processing
• dws
• gemini
• gemini-3.7-flash
• google-adk
• google-cloud
• human-in-the-loop
• invoice-processing
• nutrient-dws
• python
• vertex-ai