PROVE IT — The Evidence Graph Engine

Compliance tools manage documents. PROVE IT uses DataHub to determine whether an audit claim is actually provable from live operational evidence, not just written policy.

Comment

Inspiration

Every audit eventually reaches the same question:

"Can you prove it?"

Most organizations already have policies, documentation and dashboards.

What they don't have is a reliable way to prove that those claims are supported by live operational evidence.

Evidence is scattered across metadata catalogs, pipelines, ownership records, access reviews, CI systems and documentation. When an auditor asks a question, teams spend hours manually collecting evidence.

We wanted to build an AI agent that doesn't generate reassuring compliance text.

Instead, it determines whether a claim is actually provable.

What it does

PROVE IT turns DataHub into an evidence graph.

For every control, the agent retrieves the relevant organizational context from DataHub, including:

  • datasets
  • ownership
  • lineage
  • classifications
  • tags
  • structured metadata
  • execution evidence

It then evaluates whether the claim is supported by evidence that is:

  • present
  • current
  • owned
  • traceable
  • integrity checked
  • operational rather than merely documented

The result is one of three states:

  • ✅ PROVABLE
  • ⚠️ PARTIAL
  • ❌ UNPROVABLE

Instead of saying "you are compliant", PROVE IT explains exactly why a claim can or cannot be proved.

The resulting assessment is written back into DataHub so future users and agents inherit the updated evidence status.

Why DataHub

DataHub is not used as a simple metadata catalog.

It is the foundation of the evidence graph.

PROVE IT relies on DataHub relationships such as:

  • ownership
  • lineage
  • tags
  • structured properties
  • datasets
  • data jobs
  • metadata relationships

Without that organizational context the proof chain cannot be constructed.

How we built it

The project combines:

  • DataHub Core
  • GraphQL
  • DataHub SDK
  • deterministic evidence scoring
  • evidence dependency graphs
  • benchmark scenarios
  • Streamlit
  • Python

The scoring engine evaluates evidence using multiple dimensions including freshness, provenance, ownership, integrity and execution strength.

Unlike a pure LLM workflow, the score is deterministic and fully inspectable.

Challenges

The biggest challenge was preventing AI from creating false reassurance.

Instead of asking a language model whether a control "looks compliant", we designed a deterministic evidence engine that only evaluates connected operational evidence.

We also wanted DataHub to be central to the architecture rather than decorative.

The final implementation performs real reads from DataHub, builds evidence chains, and writes assessment results back into the metadata graph.

Results

The project currently includes:

  • live DataHub integration
  • GraphQL evidence retrieval
  • metadata write-back
  • evidence blast-radius analysis
  • security and environmental framework packs
  • adversarial benchmark suite
  • teaching labs for compliance engineering

Current benchmark:

  • 24/24 benchmark scenarios passed
  • 100% status accuracy
  • 0% false PROVABLE classifications

What's next

We see PROVE IT as an open-source foundation for continuous audit evidence rather than another compliance platform.

Future work includes additional framework packs, richer DataHub integrations, more evidence connectors, and research on graph-grounded AI for trustworthy compliance automation.

If you cannot prove it, it does not exist.

Built With

  • agents
  • ai
  • apache-2.0
  • api
  • audit
  • compliance
  • data
  • datahub
  • knowledge
  • metadata
  • open
  • python
  • sdk
  • source
  • streamlit
Share this project:

Updates