Blast Radius

Inspiration

Every data engineer has lived this: you rename or drop a column, hit merge, and three days later a finance dashboard is blank and an ML model is quietly serving garbage — and nobody connects the outage to your change. The lineage that would have warned you already exists in DataHub. The problem isn't missing metadata; it's that nobody reads the lineage by hand before every change. We wanted an agent that does that reading automatically — and does it with precision, because a tool that flags everything downstream is just alert fatigue.

What it does

Blast Radius turns DataHub lineage into a merge gate. Point it at a proposed column change — drop, rename, or retype and it:

• traces every downstream dataset, dashboard, and ML feature that actually reads that column, separates real breaks from noise, names the owners to notify, and drafts a migration plan, blocks the PR (--fail-on-breaking) so the change can't land unreviewed, and writes the verdict back into DataHub as tags, so the next person or agent inherits the knowledge. On DataHub's healthcare sample data, dropping billing_amount breaks mart_billing but clears mart_demographics; dropping age does the exact opposite proof it reads real column lineage, not just the table graph.

How we built it

The core is a deterministic engine so a verdict is never a hallucination: a breadth-first lineage traversal (with depth cap and cycle guard) feeding a precision scorer. An asset is only BREAKING when evidence ties it to the changed column, from two independent DataHub signals column-level (fine-grained) lineage and real query text. Of all downstream assets $D$, a naive tool flags all of $D$; Blast Radius flags only

$${, d \in D : \text{column-lineage}(d)\ \lor\ \text{query-ref}(d),}$$

It reads DataHub through the Agent Context Kit (search, get_lineage, get_entities, get_dataset_queries) plus the shared graph for fine-grained lineage and dashboard inputFields. Claude (claude-opus-4-8) sits on top as an optional layer that narrates the report into a plain-English PR summary, it can never override a BREAKING verdict. It ships as a CLI, a GitHub Action merge gate, and a hosted Streamlit demo.

Challenges we ran into

The live read path targeted an API that didn't exist. We had to reverse-engineer the real Agent Context Kit (DataHubContext + module-level mcp_tools) and map its GraphQL-shaped responses into our model. Column-level parity across entity types. Dashboards needed inputFields (not lineage); attaching a query to a dashboard actually crashes DataHub's listQueries (subjects must be datasets); and mlFeatureTable rejects upstreamLineage entirely, so ML lineage had to flow through MLFeature.sources. Each was a separate discovery against a live instance. Environment gremlins. The DataHub CLI's datapack/sqlalchemy ingest wouldn't build on Python 3.13, so we loaded the healthcare datapack straight from its SQLite DB via the REST emitter — and added the column-level lineage the sample ships without. Making write-back saf, idempotent, non-destructive to existing tags, and an accurate snapshot each run (stale verdicts cleared, not appended).

Accomplishments that I am proud of

Real column-level precision on real DataHub data opposite columns clear the opposite marts, on DataHub's official healthcare sample. We closed the loop. Blast Radius doesn't just read DataHub, it writes the verdict back as catalog tags, so the knowledge compounds for the next person or agent. Full parity across entity types datasets, dashboards, and ML features each modeled the way DataHub actually stores it. A safe use of an LLM: Claude explains the deterministic report but can never soften a BREAKING verdict. Ships three ways, CLI, CI merge gate, and a one-click hosted web demo.

What we learned

DataHub's metadata model is richer and more opinionated than it looks: column lineage lives on fineGrainedLineages, dashboards on inputFields, ML lineage on MLFeature.sources — each entity type wants its own mechanism.

The Agent Context Kit's MCP tools map almost 1:1 onto what an impact agent needs; reading real column lineage back out and turning it into a verdict is genuinely reliable once you speak the model's language.

An agent that only reads is half an agent — write-back is what turns a terminal report into shared, durable knowledge. What's next for Blast Radius Richer change types (type-narrowing, nullability), auto-posting the impact report as a PR comment, and writing back structured assertions so DataHub itself can circuit-break on the next run.

Built With

• agent-context-kit
• anthropic
• claude
• datahub
• mcp
• python
• streamlit