PROOFCHAIN - Trusted Document Action Engine

A correct extraction can still produce the wrong decision. PROOFCHAIN checks whether business documents agree enough to act — and stops when they don't.

Comment

Inspiration

Every business runs on document bundles, and AI can now read all of them. That part is solved.

What is not solved is the question that comes next: can we act on what these documents say?

A field can be extracted perfectly and still contradict another document. A model reporting 96% confidence has not told you whether you are permitted to approve anything. And when someone asks months later why an approval happened, the answer usually has to be reconstructed from PDFs, emails and someone's memory.

We built PROOFCHAIN around a single idea: trust is not a model output, it is a property of the workflow. So we connected every material claim to the document and page it came from, checked whether the documents agree with each other, corroborated selected facts against live public data, applied explicit decision rules, and escalated what was left to an accountable human — instead of letting a model quietly resolve it.

The result is a complete chain: documents to claims to evidence to verification to a policy gate to human review to an authorised action to a replayable record.

What it does

PROOFCHAIN is a Trusted Document Action Engine for regulated, document-heavy workflows. Our use case is supplier verification and approval.

A supplier submits five documents: company registration, tax certificate, bank confirmation, invoice, and contract. PROOFCHAIN processes them with Nutrient DWS, extracts 26 source-linked claims, groups them into 11 pieces of evidence, and compares them.

Almost everything agrees. Legal name, registration number, tax ID and address are consistent across the packet — including across three different renderings of the company name, which normalisation resolves rather than flagging.

One fact does not agree. The invoice states a bank account ending 3382. The bank confirmation letter states 9931. Both values were read at 96% and 98% confidence.

This is the point of the product. It is not an OCR failure, and no better model fixes it: the documents themselves disagree. PROOFCHAIN does not average the two, does not pick one, and does not call it fraud — a mismatch may equally be an outdated file or an ordinary change of bank. It reports conflicting evidence, drops bank-account confidence to 57%, and holds the case at 75%.

SerpApi then checks selected public claims against live web data. Here something worth watching happened: live search found real companies sharing the supplier's name, in another country, on another domain. The system examined them and declined every one. Not corroborated — which is scored as neutral, never as evidence against the supplier.

An explicit policy stops automatic approval and routes exactly one question to a person. The reviewer opens both source documents side by side, with the account numbers aligned character by character and the four differing digits marked, selects the authoritative source, and records why. Confidence recalculates to 91%. A manager — a different role — then authorises.

Afterwards, Evidence Replay answers "why did the organisation act?" from the audit trail alone: which documents were read and by which layer, what was extracted, what contradicted, what public sources said, which policy version applied, who resolved it and on what grounds, and who authorised the result.

How we built it

PROOFCHAIN is built as a set of trust boundaries rather than a pipeline of API calls.

NUTRIENT DWS — the document layer Nutrient parses each PDF and returns structured content, which an adapter normalises before it reaches our domain model. Every claim keeps its document id, page number and the raw string as written, so a reviewer opens the original document instead of trusting generated text. That path — claim back to page — is what makes the hero conflict inspectable rather than merely reported.

PERSISTENCE — behind a port, with two adapters The operational state is fifteen tables: organisations, users, suppliers, cases, documents, claims, evidence, conflicts, external checks, reviews, decisions, artifacts, policy versions and audit events. Nothing above the repository layer knows which adapter is in use, so the same 117 tests run offline against local SQL.

We defined that schema for Xano, generated it as XanoScript, wired the adapter and set up a live workspace. Seeding it exceeded the free tier's rate limit: our adapter filters client-side, so a single seed lists whole tables repeatedly and makes over a hundred requests. Server-side filtering is the fix and it is the next thing we would build. The demo therefore runs on the local adapter, and we say so rather than implying otherwise.

SERPAPI — the corroboration layer Static documents are self-asserted evidence and cannot verify themselves. Four checks run against live public data. Two design choices matter more than the integration itself: queries are refused before they leave the process if they contain anything resembling a private identifier, and "nothing found" is scored as a neutral 0.50 rather than zero — because absence of evidence is not evidence of absence, in the arithmetic and not only in the copy.

THE PROOF ENGINE — deterministic by construction A FastAPI service handles normalisation, cross-document comparison, conflict detection, confidence and the decision gate. Nothing here is generative. Evidence confidence weights source agreement at 0.75 against extraction quality at 0.15, which encodes the thesis as arithmetic: two perfectly-read documents that disagree score worse than two poorly-read documents that agree.

THE DECISION GATE — where automation stops Five states, ordered: blocked, review required, ready for approval, auto-pass. A resolved case reaches "ready for approval" rather than approving itself, because a human already intervened. Approval requires the manager role and a gate that already permits it; an analyst gets 403 and a premature approval gets 409.

FOXIT — the signing boundary The system prepares and routes an approval memorandum. It cannot sign: a signature request on an unapproved case raises before any HTTP call is made, and the signer is always a named person with an email address.

THE LANGUAGE MODEL — deliberately small It may judge whether two company names are semantically equivalent and phrase an explanation. It cannot approve, reject, resolve a conflict, set a confidence value or sign. Turning it off degrades wording and changes no decision — the gate function receives no model output on any code path.

THE INTERFACE — Next.js Five screens: cases, case overview, evidence workspace, human review, decision trail. The frontend computes nothing; it renders the gate result the engine returned, so the rules cannot drift between the API and the screen.

Challenges we ran into

The hardest problem was not extraction. It was representing uncertainty without collapsing it into a misleading binary.

We had to keep apart five things that are usually merged: extraction confidence, source consistency, external corroboration, policy compliance, and human authorisation. Two documents can each be read at 98% and still contradict each other. A conflict may be fraud, an outdated file, or an ordinary administrative change — and the system has to say which of those it knows, which is none of them.

The specification we wrote for ourselves contradicted itself in two places, and we had to decide rather than implement both. Our own rules said to block on an unresolved critical conflict, and elsewhere to route material conflicts to review. Applied together, the hero case would be blocked and no human would ever see it. We resolved it by conflict status: an open contradiction is a question for a person; only one a reviewer has examined and cannot settle becomes a block. We also added a fifth gate state, because without it a reviewed case fell back to auto-pass and approved itself, which would have made the human review theatre.

The most useful lessons came from running the thing rather than reading it. Installing from a clean checkout found five defects, three of which stopped the service from starting — including a dependency that was imported but never declared, which only worked because it happened to be installed globally on a development machine. Using the interface found that every file link returned 401, because the session token lives in localStorage and a plain anchor sends no header — which silently broke "click through to the source PDF", the interaction this product exists for.

The sharpest one was in our SerpApi client. SerpApi reports "no results for this query" through the same error field it uses for a bad key, and we treated both as provider failure. So a search that ran perfectly and found nothing was recorded as an outage and replaced with cached data. That is exactly the conflation this product exists to prevent, sitting in the integration that carries the argument. It is fixed, and a test holds the distinction.

Every one of these is recorded in docs/DECISIONS.md with the reasoning, including the ones that make us look worse.

Accomplishments that we're proud of

PROOFCHAIN never hides a material contradiction behind a polished summary. If two documents disagree, the conflict stays visible until a person settles it — and both values survive the resolution. The rejected IBAN is marked superseded, never deleted, because a trail that says a conflict was resolved without showing what it was is not a trail.

Four commitments are enforced in code rather than asserted in copy, each covered by a test:

  • Every material claim keeps its document and page. A claim without them cannot be persisted.
  • Critical contradictions are never silently resolved.
  • Generative AI cannot authorise anything. No model output reaches the decision gate on any code path.
  • Missing corroboration is not treated as evidence of falsehood. It scores 0.50, not zero.

We are most pleased with something that only appeared when we ran the checks live. Our demo supplier is fictional, and live search returned real companies sharing its name in another country. The system examined them and declined every one — then scored that absence as neutral rather than damning. That is the whole responsible-AI argument, demonstrated on data nobody controlled.

Also: the interface is honest about itself. Every document and every check is labelled live or cached, and the platform screen reports whether credentials are present rather than claiming a call succeeded. We fixed that panel specifically because it was overstating.

117 tests, no network required, about eleven seconds.

What we learned

Trustworthy AI systems need more than accurate models. They need explicit authority boundaries, source provenance, visible uncertainty, deterministic rules where determinism is appropriate, escalation paths, and a durable decision history.

The most valuable role for generative AI in this workflow turned out not to be making the decision. It was helping normalise, explain and summarise the evidence around a decision whose authority stays explicit. Our design principle became: AI assists interpretation, evidence controls action.

We also learned that a fallback which hides its reason turns a diagnosable failure into a mystery. Our integrations degrade gracefully when a provider is unavailable — which is right for a demo — but the first version discarded why, so "cached" appeared everywhere with no explanation. A graceful failure still has to be able to say what happened.

And a smaller one, learned the hard way: a sample configuration file that nothing ever executes will drift. Six keys in ours matched no setting and were ignored silently, including an API key that left a provider switched off with nothing in the logs. There is now a test that runs it.

What's next for PROOFCHAIN

Supplier verification is the wedge, not the scope. The same evidence-to-action architecture applies to procurement approvals, invoice verification, contract compliance, KYC, insurance, legal review and public-sector document workflows — anywhere a consequential action depends on documents agreeing.

The timing is not academic. The EU e-invoicing mandate, new digital-identity and anti-money-laundering rules, and electronic transferable records all land across 2026 and 2027, and each of them turns "we read the document" into "we can show why we acted on it".

Concretely: configurable workflow templates so a new document type does not need code; domain-specific evidence schemas; role-controlled policy management with versioned governance; richer signed evidence packs; and reviewer and override analytics, so the system learns which exceptions actually matter for a given workflow.

The invariant does not change: no consequential action without a traceable evidence path.

Built With

Share this project:

Updates

Submission history