Compliance Drift Sentinel

An AI agent that reads DataHub lineage to predict which downstream report field breaks when an upstream schema changes — and writes the data contract back.

Comment

Inspiration

Every data team lives in fear of the silent break: someone renames or retypes a column three hops upstream, and a regulatory report quietly starts shipping wrong numbers — no error, no alert, until an auditor finds it. DataHub already knows the lineage. What it doesn't do is reason about the blast radius of a change before it lands. We wanted an agent that answers, precisely: "if this upstream column changes, which downstream report columns break — and what contract would have caught it?" — and then writes that answer back so the next engineer inherits it.

What it does

Compliance Drift Sentinel takes a proposed upstream schema change and:

  1. Reads column-level lineage from DataHub and traces it to the exact downstream columns.
  2. Classifies severity — a drop/rename is a hard_break; a retype is a silent_break (the dangerous one: values still flow but miscompute).
  3. Narrates the impact with Gemini and drafts an enforceable data contract.
  4. Writes back to DataHub — tags, a drift_status property, and the contract — through the same MCP interface it reads from.

We demo it on a synthetic broker margin-report pipeline: retyping collateral.haircut_pct (percent → fraction) silently breaks three margin columns that still divide by 100 — a regulatory misreport with no error anywhere.

How we built it

  • DataHub v1.5 (local quickstart) as the context platform; mcp-server-datahub for both reads and write-back.
  • A deterministic impact engine (Python) that traverses fineGrainedLineages to compute the exact broken columns — the LLM never decides what breaks.
  • Gemini (gemini-flash-latest) narrates and drafts the contract, grounded strictly in the engine's facts.
  • FastAPI backend + React/Vite terminal-themed UI with a live lineage graph that lights the drift path. The impact analysis runs in-process, so the app deploys on a free tier without the 7-container DataHub stack. On the hosted demo, write-back therefore runs in demo mode — it reports exactly what it would write, since there is no DataHub in the cloud to write to. The video shows the real write-back landing in a live DataHub, and make up && make dev reproduces it locally.
  • Packaged the workflow as a reusable DataHub Skill (datahub-drift-contract).

Challenges we ran into

  • Column-precise impact. The MCP get_lineage(column=...) only resolves to the downstream dataset, not the exact columns. The real column→column edges live in the report's upstreamLineage.fineGrainedLineages aspect — we read that via the graph SDK and traverse it ourselves. That was the difference between "this table is affected" and "these three columns silently miscompute."
  • Write-back gotchas. add_tags fails unless the tag entity already exists; structured properties attach to the schemaField URN (not a column_paths argument); and mutation tools are disabled by default. Each was a live-API discovery, not something the docs spelled out.
  • A model deprecation mid-build. gemini-2.5-flash started returning 404 "no longer available to new users" — we switched to the gemini-flash-latest alias so it can't happen again.
  • Making DataHub deployable. An 8GB, 7-container stack won't fit a free tier, so we made the impact analysis computable from an in-process pipeline definition; only write-back needs the live instance.

Accomplishments that we're proud of

  • Silent-break detection that's actually column-precise — the failure mode humans miss most.
  • A grounded agent: the deterministic engine owns correctness, so Gemini can be impressive without ever hallucinating impact.
  • A genuine open-source contribution — the datahub-drift-contract skill fills a real gap in the official registry (lineage traces, quality checks, but nothing does drift → contract).
  • End-to-end and verified: detect → narrate → write-back, provable in the DataHub UI.

What we learned

  • Fine-grained lineage is where column-level truth actually lives — dataset-level lineage isn't enough for impact analysis.
  • You have to provision your vocabulary (tags, structured properties) before an agent can write it back.
  • Keeping the LLM out of the correctness path — deterministic core, LLM for narration — is what makes an agentic tool trustworthy.

What's next for Compliance Drift Sentinel

  • Run the Sentinel as a CI gate on every schema-change PR.
  • Emit the drafted contract as a native DataHub assertion.
  • Upstream the datahub-drift-contract skill to the official registry.

Built with

datahub · mcp-server-datahub · python · fastapi · react · vite · gemini · docker

Built With

  • ai-agent
  • data-contracts
  • data-governance
  • data-lineage
  • data-quality
  • datahub
  • docker
  • fastapi
  • gemini
  • mcp
  • model-context-protocol
  • playwright
  • pydantic
  • pytest
  • python
  • react
  • render
  • typescript
  • vercel
  • vite
Share this project:

Updates