Fuse: the blast-radius agent for DataHub

Inspiration

Every data team has the same scar: a small schema change ships, CI passes, and something downstream is quietly wrong for days. The blast radius is knowable, since it sits in the lineage graph, but nobody has time to walk it by hand, and the ML half is the part everyone forgets. While building this we measured something worse: DataHub's get_lineage returns the model downstream of a dataset, but not the deployment serving traffic, and it can't tell you which feature broke. It returns the same four features whether you dropped one of them or none.

What it does

Fuse turns a diff into a decision and a fix. It parses the schema change out of your data repo with sqlglot, resolves it to DataHub entities, walks downstream lineage, and then reads the ML aspects directly, following MLFeature.sources → MLFeatureTable → MLModel → deployments, because generic lineage doesn't cover that path. It scores the blast radius with a deterministic rule engine, generates the remediation (rewritten models, a compatibility view, contract tests, a migration plan), opens a PR with an owner-tagged impact report, and writes the verdict back into DataHub as tags, structured properties and a saved document.

On our test catalog, dropping credit_limit from a dbt model surfaces the feature built on it (BREAKING, 90), the churn model consuming it (67), and the production deployment (64). Lineage alone does not connect the last two to that column.

Evaluate it in two minutes, without installing DataHub

There is nothing to host, by design. Every DataHub response and every model response is recorded, so the whole pipeline runs from a clean clone with no Docker, no catalog and no API key:

pip install -e . && fuse replay examples/03-ml-feature-break

It regenerates the committed artifacts byte for byte. To judge output quality without running anything, read examples/03-ml-feature-break in this order: impact-report.md for the verdict and the arithmetic behind every score, generated/models/compat/customers_compat.sql for the remediation, then run.log and generated/writeback.json for the node-by-node trace and exactly which urns were written back to the catalog.

Two CI workflows keep this honest. ci runs 171 tests on Python 3.10-3.12. selftest installs the project from a clean checkout on a runner, replays the ML scenario with no catalog and no key, and asserts the regenerated report still reaches prod-retention-service, so the exact path a judge takes is itself covered by CI.

How we built it

LangGraph orchestrates nine nodes over typed state. Reads and writes go through the DataHub MCP Server via langchain-mcp-adapters; ML entity discovery uses GMS GraphQL and ML aspects the typed Python SDK, because the MCP surface exposes neither. Change parsing and validation use sqlglot. Scoring is deterministic and configured in rules.yaml: nothing a language model says ever becomes a number. Every DataHub call and every model response is recorded, so the committed examples replay byte-for-byte offline.

Challenges

Keeping generated SQL trustworthy. The generator found a new way to be subtly wrong at almost every step: nulling out a dropped column in a rewritten model, so the shape survives and the data silently doesn't; casting a narrowed column back to its old type, hiding the truncation; emitting a column twice; deleting a column that had only been retyped. Each one now has a deterministic guard and a test, and each guard had to learn where it does not apply, because a compatibility view preserves that same shape on purpose. The interesting part is that the validator originally caught only wrong content, never missing content. It had been built entirely from prohibitions.

Accomplishments

An end-to-end read → act → write loop that produces merge-ready code; ML impact analysis that reaches a production deployment through a path lineage doesn't expose; and a demo that reproduces exactly, offline, with no API key. 171 tests, including replays of every committed example.

What we learned

Metadata is only worth what you do with it, and a catalog's API surface is not the same thing as its data model. Three of the gaps we hit are filed upstream. Also: with the schema grounding and guards in place, a free 9B open-weight model produced the same correct output as a 120B one. The deterministic layer does the correctness work, so the model doesn't have to be large or expensive.

What's next

More dialects, semantic diffing for logic rather than schema changes, and a policy layer so teams can encode their own migration standards.

Built With

• datahub
• github-jobs
• gms
• graphql
• jinja
• langgraph
• mcp
• openrouter
• sdk
• server
• sqlglot