TrustLedger

AI receipt extraction verified by independent math, not just AI confidence — catches errors confidence scores alone miss.

Comment

Inspiration

AI receipt/invoice extraction tools generally do one of two things: overpromise full automation and quietly get things wrong, or dump raw OCR text you still have to clean up yourself. We built TrustLedger to do something different: extract everything automatically, but actually know what it doesn't know.

What it does

TrustLedger processes receipts through Nutrient DWS's schema-guided /extraction/extract endpoint to pull vendor, date, line items, and totals as typed JSON, with per-field confidence and citation metadata. Every result then passes through two independent review layers:

  1. DWS's own confidence/citation scoring — flags fields the model couldn't ground in the source document.
  2. A reconciliation layer we built on top — plain arithmetic checks that don't rely on the AI's self-assessment at all: does qty × unit price equal the line total? Does the sum of line items equal the subtotal? Does subtotal + tax equal the total? Does the vendor/item name contain garbled or look-alike characters?

Anything flagged surfaces in a review UI next to the original receipt image, with the specific reason shown live. A human corrects it in seconds; everything else auto-approves. Every correction — what the AI extracted, what a human changed, when — is logged to an audit trail, and the final data exports as a clean CSV.

Why the second layer matters — this is the finding, not just a feature

We tested the full pipeline against 20 receipts (synthetic, spanning grocery/restaurant/gas/hotel categories, 30% deliberately degraded). Results:

  • 100% extraction success rate (20/20)
  • 65% auto-approved with no issues found by either layer
  • 35% (7 receipts) flagged for human review
    • DWS's own confidence scoring caught only 2 of those 7 on its own
    • Our reconciliation layer caught 5 of the 7 that DWS's confidence scoring missed entirely

The clearest example: one receipt had badly corrupted line items — truncated item names, a dropped decimal point — but the extracted top-level subtotal and total still happened to match ground truth exactly, likely read directly off the receipt's printed summary line. DWS's confidence scoring saw nothing wrong. Our reconciliation check caught it instantly: the line items summed to $2,294.64, nowhere close to the $516.79 subtotal.

Trusting the AI's own confidence score alone would have missed most of the real errors in our test set.

How we built it

  • Nutrient DWS Data Extraction API — schema-guided extraction (understand mode), per-field confidence + citation metadata
  • Python — extraction pipeline, reconciliation logic
  • Streamlit — human review UI with live field validation, one-click auto-fix for garbled text, and a receipt viewer that opens in a separate browser window
  • Pillow + Faker — synthetic receipt/ground-truth generator for testing at scale

Challenges we ran into

Along the way we found and fixed real bugs in our own logic, not just the AI's: a naive percentage-only tolerance check let a flat $10 error slip past undetected on a large receipt, until we switched to an amount-scaled tolerance. We also found that human corrections needed the same reconciliation re-check as the AI's original output — a person can just as easily retype a value wrong, or miss an invisible look-alike Unicode character that renders identically to the correct one on screen.

What's next

  • Testing against real-world scanned receipts (SROIE/CORD datasets)
  • Tuning confidence thresholds per document category using a larger labeled sample
  • Extending garbled-text detection with typo/fuzzy-matching against known vendor names

Built With

Share this project:

Updates

Submission history