blastradius

Inspiration

Someone opens a pull request that drops a column. Review approves it, because the diff looks fine and nobody can hold the whole warehouse in their head. Three days later an executive dashboard is blank and an ML feature table is full of nulls.

The information needed to prevent that already exists in DataHub. Nothing consults it at the moment the decision is made.

What it does

blastradius takes a proposed schema change and answers the question code review cannot: what does this column actually feed?

It walks DataHub column-level lineage, finds every downstream table, dashboard, job and ML asset that reads the changed column, resolves the owner of each, ranks them by who feels the breakage, writes a markdown report sized for a pull request comment, and tags the impacted assets back into DataHub.

Real output against a live DataHub:

$ blastradius scan '...raw.orders,PROD)' --drop user_id

 severity   asset                    kind        columns    hops   owners
 breaking   Revenue Overview         dashboard   buyer_id   2      unowned
 breaking   analytics.orders_daily   dataset     buyer_id   1      analytics
 breaking   ml.customer_features     dataset     user_id    1      ml-team

 3 breaking, 0 risky, 0 downstream but unaffected

Follow the top row. The column is user_id upstream, but the dashboard renders it as buyer_id, and nobody owns that dashboard. Grepping the repository for user_id finds neither the rename nor the dashboard.

How it uses DataHub

Not as a search box. Five distinct parts of the metadata graph, each doing work that changes the answer:

what	why it matters
column-level lineage (`searchAcrossLineage` on a `schemaField` URN)	proves an asset reads this column, which separates breaking from merely nearby
table-level lineage	catches assets whose connector never emitted fine-grained lineage, so gaps are reported rather than hidden
ownership	turns "something breaks" into "these three people need telling"
`inputFields` on dashboards and charts	closes the last hop into the BI layer, which lineage traversal alone cannot cross
write-back (tags, institutional memory links)	the verdict lives with the data, not in a PR thread nobody reads again

The part I am most pleased with

DataHub fine-grained lineage links warehouse columns to warehouse columns. Dashboards and charts instead declare what they render in a separate inputFields aspect, and searchAcrossLineage does not traverse it.

So the graph knows analytics.orders_daily.buyer_id is affected, and separately knows the Revenue Overview dashboard renders analytics.orders_daily.buyer_id, but nothing joins those two facts.

blastradius bridges that hop. It is the difference between reporting "a table changed" and reporting "this named dashboard, owned by nobody, goes blank".

Being honest about evidence

The easy version of this tool reports everything downstream and calls it all breaking. That produces a scary number and gets ignored within a week. Instead every impact is labelled by the evidence it rests on:

evidence	severity
downstream reads the column, and it is dropped or renamed	breaking
downstream reads the column, and it is retyped	risky
downstream of the table, no column-level link recorded	info

A retype is risky rather than breaking because the reference still resolves by name; it fails later, in the result, not immediately. And info means not proven, not safe: missing fine-grained lineage is common, and the report says so instead of implying the change is clear.

How I built it

Python, four dependencies in the core and no LLM framework. The impact logic is separated from the DataHub client so it is testable without a running instance: 23 unit tests, plus scripts/verify.sh which runs all eight steps of the loop against a real local DataHub and fails loudly if any of them regress.

Challenges I ran into

Three things that surfaced by running it:

The lineage degree filter is an enum, not a range. DataHub accepts "1", "2" and "3+", and rejects "3" outright. Every scan failed until the hop filter was expressed the way the API actually models it. This one is on me: it is stated plainly in the GraphQL best practices page, which I reached only after debugging it. Worth the embarrassment of writing down, because the failure looks like a bad request rather than a wrong vocabulary.
Selecting properties { name } across several entity types in one GraphQL selection set is rejected, because the name fields have different nullability shapes and cannot be merged under one response key. The specific culprit is MLModelProperties.name, which is String while the dataset, dashboard, chart and data job equivalents are all String!. The fix is per-type aliases plus a shared fragment for ownership.
A severity bug I introduced and caught. The BI bridge initially applied the worst severity across all changes to any dashboard it upgraded, so a dashboard rendering only a retyped column was reported as breaking because some unrelated column was dropped. Now each impacted column remembers which change endangered it, with a regression test that fails on the old behaviour.

What I learned

That the interesting metadata problem is not retrieval, it is joining facts the graph holds separately. DataHub knew everything needed to predict the broken dashboard. It just never connected column lineage to BI input fields, and that one missing join is the whole product.

Using it from an agent

The analysis is exposed over MCP as well as on the command line, because the two callers arrive with different information. A human types --drop user_id because they already know what they are changing. An agent reviewing a pull request has a diff and has to work out what it endangers before it can say anything useful, so it gets list_columns to reconcile the diff against the catalogue, scan_schema_change to trace the damage, and record_impact to write the verdict back.

Reading and writing are separate tools rather than one tool with a flag. An agent should be able to call the read as often as it likes while exploring, and should have to choose the catalogue mutation on purpose. record_impact also re-runs the scan instead of trusting impacts handed back to it, so a stale or edited payload cannot write a claim into DataHub that no lineage walk ever supported.

It is an optional extra, pip install blastradius[mcp], so the core keeps its four dependencies. Verified by connecting a real MCP client over stdio: three tools advertised, a tool call round-tripped.

Open-source contributions to DataHub

Two of the three findings above went back upstream, because a workaround living only in this repository helps nobody else who hits the same wall.

datahub#18791, issue. searchAcrossLineage does not join column lineage to dashboard inputFields. DataHub holds both halves of that join: fine-grained lineage proves raw.orders.user_id feeds analytics.orders_daily.buyer_id, and the dashboard's inputFields aspect declares it renders buyer_id. Nothing connects them, so a column-level impact query reaches the warehouse boundary and stops, precisely where a human is looking at the result. The report carries a minimal reproduction and notes that the seed emits inputFields exactly as the Looker and Tableau connectors do, so this is not an artefact of synthetic data. _bridge_bi_input_fields() in this repository is the workaround, and at roughly thirty lines it suggests the join is cheap wherever it properly belongs.

datahub#18792, pull request. Documents the properties aliasing requirement in the GraphQL best practices page, naming MLModelProperties.name as the nullability outlier so the next person can check it against the schema rather than discovering it at validation time.

The degree filter finding was deliberately not filed. It is already documented, and a hackathon is a bad reason to open an issue that wastes a maintainer's afternoon.

Using it in CI

blastradius scan "$DATASET_URN" --drop user_id \
  --out impact.md --write-back --pr-url "$PR_URL" --fail-on-breaking

--fail-on-breaking exits non-zero so the merge is blocked, --out writes the PR comment, and --write-back records the verdict in DataHub. A working GitHub Actions workflow is in examples/github-workflow.yml.

What is next

Deriving the change automatically from a dbt manifest diff, so nobody types --drop user_id by hand. It is deliberately left out rather than shipped half-working.