ProofDesk

Inspiration

I did not come up with this project before seeing the scope Nutrient required but it was a new topic I found interesting and the extent to which autonomous agents could be trusted with sensitive information

What it does

It is an evidence-gated document execution system with audit chain so you can see exactly where and why decisions are made. I tried a lot of experiments on how to maximise auto sign rate while keeping mistakes to 0% or 1% but still found overwhelmingly that human review is needed at least 40% of the time especially with high risk documents.

How we built it

The pipeline has five stages:

1. Extraction — Nutrient DWS pulls grounded evidence from uploaded PDFs. Every field carries value, confidence, source page, and bounding box.
2. Cross-document verification — Reconciliation checks facts across documents. When a procurement request contradicts a vendor quote, the system detects it.
3. Risk classification — 6 calibrated algorithms estimate decision risk. Conformal risk control provides mathematically guaranteed error rates.
4. SignatureGate — 6 server-side enforced conditions: state is PREPARED, no unresolved blockers, human approval present, structured record exists, artifact hash verified, calibrated score ≥ threshold. The agent cannot negotiate any of these.
5. Audit trail — Hash-chained events with Merkle proofs. Every decision, every extraction, every human review is recorded and tamper-evident. Human decisions feed back into calibration. Over time, the system learns where this organization can safely automate.

Challenges we ran into

Confidence calibration. Raw extraction confidence from Nutrient is useful but not sufficient. We needed calibrated risk scores that account for document complexity, field importance, and cross-document consistency. Building 6 calibrated algorithms — from basic thresholds to conformal risk control — was the hardest technical work.

Accomplishments that we're proud of

Approaching a new problem area and thinking creatively

What we learned

Humans should remain responsible for AI mistakes

What's next for ProofDesk

Multi-format ingestion — extend beyond PDFs to Word docs, spreadsheets, scanned images Batch processing — run the full pipeline across document collections with automatic contradiction detection Learning loop refinement — improve the human-feedback calibration with more data points Regulatory compliance templates — pre-built SignatureGate configurations for specific compliance requirements (SOX, GDPR, HIPAA) API marketplace — let other applications integrate ProofDesk as a decision-gating service

Built With

• cloudflare-pages
• conformal-prediction
• fastapi
• foxit-pdf
• merkle-proofs
• mit
• nutrient-dws
• python
• sha-256