Datahub Impact Radar

The agentic copilot that turns DataHub lineage into a risk score — and writes the decision back.

Comment

Inspiration## Inspiration

Every data engineer has lived this moment: you go to rename a column, drop a field, or deprecate a table, and the honest answer to "what could this break?" is manually crawl the lineage graph and hope you don't miss a dashboard. DataHub already has the metadata to answer that question — schemas, lineage edges, ownership — but it's scattered across separate views, and even when you find it, the decision you made evaporates the moment the Slack thread scrolls past. We wanted to close that loop: turn "what breaks if I rename this?" into a single plain-English question, and make sure the answer outlives the conversation by writing it back into the catalog itself.

Inspiration

The spark was a very ordinary, very expensive kind of mistake: someone renames a column — customer_id to cust_uid, say — because it's a two-minute change in isolation, and three dashboards and a nightly pipeline break somewhere downstream that nobody thought to check. The information needed to prevent that already exists. DataHub knows the lineage, the ownership, the dashboards that read from that table. What's missing isn't data, it's a fast path from "I'm about to make this change" to "here's exactly what breaks, how bad it is, and what to do about it" — in the time it takes to read a Slack message, not the time it takes to click through a lineage graph by hand.

That gap — rich context that exists but isn't assembled into a decision — is exactly the shape of problem the hackathon's framing calls out: agents that have complete context can do work a data engineer would otherwise have to do manually. So we built the agent we wished existed: describe the change in plain English, get a trustworthy answer in seconds, and leave the DataHub catalog richer than you found it.

What it does

You type something like "Rename customer_id to cust_uid in the orders table." Impact Radar:

  1. Resolves the target asset against DataHub (search + get_entities), disambiguating if the phrase is ambiguous.
  2. Traces the blast radius via DataHub's real lineage graph (get_lineage) — dependent pipelines, dashboards, and derivative datasets, not a guess.
  3. Scores risk with a deterministic, explainable heuristic — Low / Medium / High, always paired with the named reasons that produced it. No opaque model in the loop for this step, on purpose.
  4. Recommends a mitigation plan tailored to the change type: alias period, owner notification, dashboard validation, delayed cutover.
  5. Writes the decision back into DataHub on explicit confirmation, so the next person (or agent) who looks at that asset inherits the judgment instead of having to redo it or find it in a chat log six months later.

The whole analysis path — steps 1–4 — runs in single-digit seconds. The write-back step is deliberately the one place a real agent runs: it drives DataHub's own datahub-enrich skill through the Claude Agent SDK, with DataHub's MCP tools as its only means of touching the catalog.

How we built it

The backend (services/impact-engine, FastAPI/Python) never talks to DataHub directly — it goes exclusively through DataHub's own mcp-server-datahub MCP server, composing its documented search, get_entities, get_lineage, and mutation tools rather than reimplementing any of that surface. Every pipeline stage — intent parsing, asset resolution, impact analysis, risk scoring, recommendations — is deterministic, testable code with its own unit tests against recorded fixtures, plus a live integration test run against a seeded DataHub instance for the canonical rename scenario.

Write-back is the exception, deliberately: it's the one component that is a real agent. writeback_service/ drives datahub-enrich's system prompt as an actual Claude Agent SDK query() run, with DataHub's MCP server registered as its tool source, and streams its tool-by-tool activity to the UI live — so a judge watching the demo sees real ToolSearch → update_description → get_entities calls happen, not a spinner.

The frontend (services/frontend, React/TypeScript) is intentionally narrow: four screens (input → impact view → action plan → write-back result), built so a judge understands the result in under 15 seconds — a constraint we held onto throughout rather than letting the graph view grow more interactive complexity than the decision actually needs.

Because judging explicitly rewards writing back to the graph, not just reading it, we didn't stop at one write-back mechanism. write_back() independently snapshots the entity via get_entities before and after and diffs the two, so the result the UI shows is verified, not just the agent's self-report of what it thinks it did.

Challenges we faced

DataHub's write surface is narrower than it looks. add_structured_properties only sets values on already-defined structured properties, and none of the five seeded on this instance (costCenter, dataFreshnessSla, etc.) fit a planned-change note. add_tags rejects tag URNs that aren't pre-existing Tag entities. No MCP tool defines new properties or tags. In practice, the one mechanism that reliably persists a durable, freeform note is update_description (append-only) — so we designed the write-back model around that reality instead of the mechanism we originally assumed we'd use, and verified every claim live rather than trusting documentation.

Never trust the agent's own report. Early on, a write-back run could report success even when the mutation didn't actually land — the agent's final message isn't proof. We fixed this by making write_back() treat the agent's report as a hypothesis, not a fact: it independently diffs a before/after get_entities snapshot and only reports what it can actually observe changed.

Lineage resolution is lossier at finer grain. Table-level downstream lineage for orders surfaces 41 assets including dashboards and derived tables; column-scoped lineage (column="customer_id") returns only 5 pipeline-hop assets and silently drops the dashboards. If we'd used column-level lineage as the primary signal, the demo scenario would have under-reported its own headline risk. We use table-level lineage as the impact engine's source of truth and column-level only as a precision annotation layered on top.

Keeping risk scoring honest. It would have been faster to hand the LLM the lineage graph and ask it to eyeball a risk tier. We didn't, on purpose — the PRD and both root and service CLAUDE.md files treat this as a hard constraint. Every risk tier has to carry the specific signals (business-facing downstream assets, hop distance, asset criticality) that produced it, which meant building an actual deterministic, auditable scoring function instead of outsourcing the judgment.

Reproducing the standing acceptance test, not just unit tests. The canonical scenario — rename customer_id to cust_uid in orders — is the one thing that must never break, so it's a live integration test run against the real seeded instance after any change touching more than one component, not just something covered by fixture-based unit tests in isolation.

What we learned

The most durable lesson was architectural: the parts of this system that benefit from being an "agent" and the parts that don't are not the same parts, and conflating them is a mistake. Resolution, lineage traversal, and risk scoring are better as deterministic, inspectable code — an LLM free-associating a risk tier is a liability, not a feature, when the whole point is that a judge (or a data engineer) can trust why the system said "High risk" in under fifteen seconds. Write-back is the opposite case: it benefits from being a genuine agent because the task — drive a skill, interpret its tool results, decide what to verify — is exactly the kind of multi-step tool use agents are good at. Knowing where to draw that line mattered more than any individual implementation detail.

We also came away with a much more concrete sense of what "meaningful use of DataHub" means in practice, past the marketing description: it's the difference between calling get_lineage once to decorate a UI, and building a system where every downstream write is independently verified against the graph before it's presented as fact. That distinction shaped the write-back verification design more than anything else in the project.

Finally, contributing the same change-impact judgment upstream as a standalone datahub-change-impact Claude Code skill (currently open as PR #72 against datahub-project/datahub-skills) forced us to separate the generalizable workflow — search → lineage → risk → enrich — from the parts that are genuinely specific to Impact Radar's product surface. That separation made the app's own architecture cleaner too.

Built With

  • claude-agent-sdk
  • fastapi
  • mcp-server-datahub
  • model-context-protocol
  • openapi
  • pydantic
  • pytest
  • python
  • react
  • typescript
  • uv
  • uvicorn
  • vite
  • vitest
Share this project:

Updates