AssureGraph

From lineage impact to verified change readiness.

Comment

Inspiration

Lineage answers "what could this change affect?" It does not answer the release question that usually follows: "do we have valid evidence that every affected behavior is safe for this exact revision?" A green test from another schema snapshot, field, environment, or source revision can be more dangerous than an obvious failure because it looks reassuring.

AssureGraph closes that gap. It treats DataHub as the trusted context graph, test artifacts as version-bound evidence, models as proposal generators, and human review as authorization—not as a substitute for proof.

What it does

The synthetic demo changes customer_orders.customer_tier from free-form text to a controlled vocabulary.

  1. AssureGraph reads the source schema, field-level lineage, ownership, and governed properties from DataHub OSS.
  2. It resolves four downstream consumers and constructs four independently decidable proof obligations.
  3. The baseline JUnit report proves the schema binding but leaves three behavioral gaps: legacy value handling, revenue reconciliation, and downstream filter behavior.
  4. A DeepSeek Flash agent asynchronously proposes a focused contract suite. Its output is labeled advisory and cannot change evidence, approval, execution, or readiness.
  5. Execution is rejected until a human records an append-only approval bound to the exact target version.
  6. The single allow-listed synthetic action emits fresh JUnit evidence. AssureGraph validates its asset, field, schema, revision, environment, run, and artifact digest bindings.
  7. Only after all four obligations are satisfied does readiness become ready. AssureGraph writes four CUSTOM assertion results and per-key readiness properties back to DataHub.

The result is a closed operational loop: DataHub context → proof obligations → evidence gaps → reviewed action → fresh evidence → verified DataHub write-back.

How we built it

AssureGraph is a clean-room Apache-2.0 Python application running against DataHub OSS 1.6.0. It uses explicit GMS REST aspects for the synthetic catalog and field lineage, GraphQL for lineage impact and CUSTOM assertions, and JSON Patch for non-destructive readiness properties.

The domain and application layers are deterministic and network-free. A hardened JUnit/pytest XML connector rejects DTD/XXE input, enforces size and mapping limits, produces stable evidence IDs and digests, and reports duplicate or unmapped cases. The FastAPI workbench renders immediately while the proposal-only model analysis is prewarmed and cached. Docker runs read-only with all Linux capabilities dropped and no-new-privileges enabled.

The repository includes a one-command quickstart, six synthetic datasets, five field-lineage edges, baseline and remediated evidence, the full accepted assurance policy, 85 automated tests on Python 3.11/3.12, and a public-boundary verifier.

Challenges we ran into

DataHub OSS CUSTOM assertions have important real-world contract details. Result-reportable assertions must be dataset-scoped in OSS 1.6; field bindings therefore live in structured assertion logic. Newly created assertions also have a short eventual-consistency window before result reporting succeeds. AssureGraph retries only DataHub's exact not-yet-associated error on a bounded schedule and fails immediately for validation or authorization failures.

We also had to keep three concepts separate: impact is not risk, review is not truth, and a passing artifact is not evidence until its exact version binding is verified.

Accomplishments that we are proud of

  • Real end-to-end DataHub OSS reads and writes, not mocked screenshots.
  • Four downstream assets become four explicit, auditable proof obligations.
  • Baseline 1/4 coverage and 3 gaps close to 4/4 only through fresh evidence.
  • Human approval unlocks one named action but never changes readiness by itself.
  • Four deterministic CUSTOM assertions and non-destructive readiness properties are written back.
  • An append-only, idempotent, hash-linked audit chain makes every transition reproducible.
  • The same proof-gap semantics were contributed to the public DataHub Skills repository in issue #95 and PR #97.

What we learned

The strongest agent boundary is not a bigger prompt. It is a contract that says which context is authoritative, how evidence is bound, which transitions are deterministic, what requires approval, and which state can be written back. DataHub provides the shared context surface; AssureGraph adds the governed evidence loop.

What's next

The next production steps are durable multi-user workflow storage, enterprise identity and authorization, CI-provider adapters, richer evidence connectors, incremental re-review when lineage or policy changes, and a DataHub-native presentation for proof obligations and historical readiness. The current submission intentionally keeps one high-fidelity vertical slice instead of claiming an unfinished general platform.

Built With

  • datahub-oss
  • docker-compose
  • fastapi
  • graphql
  • junit
  • pydantic
  • python
Share this project:

Updates