Inspiration

A payment dispute is, at its core, a question about what actually happened.

A customer pays for a digital product. The payment is captured. A licence is issued. An access link is generated. But when a dispute arrives, the seller is suddenly expected to reconstruct the entire transaction and prove every claim with reliable records.

And that is where things get complicated.

Unlike physical goods, digital products leave behind fragmented trails of evidence: payment records, order events, licence records, access logs and delivery notifications. These records may all exist, but they rarely tell one connected story.

More importantly, they do not all prove the same thing.

  • A successful payment does not prove delivery.
  • A generated licence does not prove that it was downloaded.
  • An access link being created does not establish that the customer used it.

We realised that the real problem wasn't a lack of data. It was the lack of an intelligent system that could distinguish what the evidence proves from what we merely assume.

That is the idea behind ProofLedger: turn scattered transaction records into evidence-backed conclusions, without allowing AI to invent certainty.


What it does

ProofLedger is an AI-powered evidence verification assistant built for digital goods sellers.

It takes factual claims and evaluates them against their underlying source records. Instead of simply collecting documents or generating persuasive explanations, it determines how strongly the available evidence supports each claim.

Every verification produces one of three outcomes:

  • SUPPORTED — The available source evidence supports the claim.
  • CONTRADICTED — The available evidence conflicts with the claim.
  • INSUFFICIENT EVIDENCE — The available records are not sufficient to establish a claim.

The distinction matters. ProofLedger does not interpret missing evidence as proof of failure, or a payment confirmation as proof of product delivery.

It also validates transaction relationships. Evidence belonging to Order B cannot automatically be used to establish what happened with Order A.

Once claims have been evaluated, relevant records can be organised into structured, source-linked evidence packets, giving sellers a clearer picture of what they can substantiate, what conflicts with their account and what is still missing.

The seller remains in control: ProofLedger assists with evidence preparation and requires human approval for consequential submission actions.


How we built it

We built ProofLedger around one architectural principle:

Let deterministic code verify what can be checked precisely, and let AI reason about what requires language understanding.

Our backend is built with Python and FastAPI, with a modular evidence-verification pipeline.

1. Deterministic Validation Engine

The first layer checks transaction IDs, order relationships, payment states, timestamps and source consistency. It enforces strict record linkage so that unrelated transactions cannot silently become evidence for one another.

2. Natural Language Inference

The second layer uses a DeBERTa-based NLI model to evaluate the relationship between a factual claim and its supporting evidence. It helps classify whether a source supports, contradicts or fails to establish a claim.

3. PayPal Integration

We implemented PayPal REST API functionality for order creation, payment capture, transaction retrieval and dispute-related workflows. Our webhook processing layer handles relevant payment and dispute lifecycle events and uses PayPal's webhook signature-verification mechanism before processing incoming events.

4. Persistent and Tamper-Evident Records

SQLite stores transaction and webhook records. Alongside it, we maintain a SHA-256 hash-linked event ledger, making subsequent changes to recorded entries detectable and providing a traceable event history.

5. Evidence Compilation and Human Approval

The evidence compiler organises relevant records into structured, source-linked packets. A human approval gate keeps the seller in control of any consequential action. ProofLedger does not automatically submit dispute evidence.

The result is a hybrid system where AI interpretation is supported by deterministic checks, traceable sources and explicit decision boundaries.


Challenges we ran into

1. Distinguishing related events from equivalent events

The biggest challenge was teaching the system that two events can be related without being equivalent.

A payment can be completed while delivery remains unverified. A licence can exist without a download. An order can be approved while capture is still pending.

We had to encode these distinctions into the validation logic rather than allowing the model to infer beyond the available records.

2. Preventing AI from becoming confidently wrong

During our model experiments, we observed that high validation performance did not necessarily translate into reliable held-out performance. We explored different model configurations and used synthetic evidence benchmarks to examine generalisation, class-level recall and false SUPPORTED predictions.

This changed our approach: the system needed to treat insufficient evidence as a first-class outcome, not an inconvenient edge case.

3. Building dependable payment infrastructure

We also had to handle the engineering details that make an evidence system dependable: duplicate webhook deliveries, signature verification, exact transaction correlation, persistent event processing and hash-chain integrity.

One important lesson was that a technically successful API response is not the same as an independently verified end-to-end business event. That distinction continues to guide our testing.


Accomplishments that we're proud of

  • Built a hybrid AI verification pipeline combining deterministic validation and Natural Language Inference.
  • Implemented a three-outcome classification system that explicitly accounts for insufficient evidence.
  • Developed strict transaction-linkage checks to prevent cross-order evidence contamination.
  • Integrated PayPal REST API workflows and implemented a webhook-processing architecture with signature verification.
  • Built persistent webhook processing with duplicate-event protection and SQLite storage.
  • Implemented a SHA-256 hash-linked event ledger and structured evidence compilation.
  • Established human approval as a mandatory boundary before consequential evidence-submission actions.
  • Expanded our automated unit and API test suite to 98 passing tests in our latest reported run.

We are particularly proud of the decisions the system is designed not to make: it does not assume that payment means delivery, does not treat missing records as definitive proof, and does not allow an AI-generated conclusion to replace its underlying evidence.

Our real PayPal Sandbox end-to-end lifecycle verification is the next validation milestone; the automated test results should not be confused with a completed live Sandbox transaction.


What we learned

Building ProofLedger taught us that trust in AI cannot come from confident language or impressive-looking outputs.

It comes from traceability, validation and knowing when not to make a decision.

We learned that deterministic rules and AI reasoning are not competing approaches. Used together, they can compensate for each other's weaknesses.

We also learned that an evidence system must preserve the distinction between an event happening, a record describing that event and a claim being established by that record.

Perhaps our biggest learning was this:

A trustworthy AI system is not one that always has an answer. It is one that knows when the evidence is not enough.


What's next for ProofLedger

We want to take ProofLedger from a working verification backend towards a complete evidence intelligence platform for digital commerce.

Our next priorities are:

  • Complete PayPal Sandbox lifecycle testing: Including authenticated webhook delivery and end-to-end transaction reconciliation.
  • Improve AI reliability: Improve NLI generalisation, insufficient-evidence recall and calibration to reduce overconfident conclusions.
  • Build a polished seller-facing interface: For claims, evidence review, verification results and evidence packet management.
  • Expand integrations: Connect digital delivery, licence management and access-control systems.
  • Strengthen evidence provenance: Develop richer audit capabilities so sellers can trace every conclusion back to its original source.
  • Real-world evaluation: Evaluate the system against more diverse, carefully validated evidence scenarios as suitable data becomes available.

Our long-term vision is to make evidence verification a reliable layer of digital commerce — one that helps sellers explain what happened through records they can actually substantiate.

We don't want AI to write a better story for a seller. We want it to help establish what the evidence can honestly say.

That's ProofLedger.

Built With

Share this project:

Updates

Submission history