Faultline

Know your PR's blast radius before you merge. Faultline asks DataHub which downstream assets actually read each changed column, then writes the migrations for what breaks.

Comment

Inspiration

Every data team has the same story. Someone renames a column in a dbt model, the diff is four lines, it passes review, it merges. Forty hours later three dashboards are blank and a churn model has been scoring on nulls all week.

What makes it maddening is that the information needed to prevent it already existed. It was in DataHub the whole time; who consumes this table, which of them are production-critical, which carry PII. Nobody looked, because looking meant leaving the pull request, opening another tool, and reading a lineage graph by eye while a reviewer waited.

So the problem isn't missing metadata. It's that the metadata isn't where the decision gets made. Faultline puts it there.

What it does

At pull request time, autonomously:

  1. Reads the diff with a real SQL parser and works out what actually changed: crucially, telling a rename apart from a drop by comparing source expressions, not names. cust_email and cust_email as email_address are the same expression under two names, which is mechanically fixable; a column whose source expression vanishes is a drop, which usually isn't.

  2. Asks DataHub which assets read each changed column, not which sit downstream. On our demo change: 25 assets are downstream, 21 read a changed column, and 4 are ruled out. Reporting those 4 as safe is what makes the other 21 credible.

  3. Rates the change deterministically from facts a practitioner acts on - PII tags, affected charts and dashboards, ownership gaps - with every point of severity carrying a sentence explaining it. No model is consulted for the rating.

  4. Writes the migration for downstream models that break, grounded in their real SQL and DataHub's real schema.

  5. Publishes the assessment back into DataHub as a document on the changed model, and tags all 21 affected assets so the next person who opens one sees the pending risk.

How we built it

Five units, each independently testable: sqldiff (pure, no network), graph (DataHub), impact (pure, deterministic), patch (Claude Opus 5), writeback (DataHub).

Everything DataHub-facing goes through the DataHub MCP Server over stdio:

MCP tool What Faultline does with it
search Resolve a changed dbt model to its URN
get_entities Owners, tags, glossary terms, schema
get_lineage Walk downstream hop by hop, recording distance
get_lineage with column The core of the tool — which assets read this specific column
get_dataset_queries Real consumer SQL, used to ground generated migrations
save_document Publish the assessment back onto the model
add_tags Mark every affected asset with the pending risk

Three decisions we'd defend:

Generated code is verified, not trusted. The model returns whole updated files and difflib derives the diff, so a patch can never disagree with the file it claims to change. Each patch is then re-parsed with the same parser that found the break, if it still reads a removed column it's reported unverified rather than shipped. Reading and emitting are distinguished:

-    o.order_status,
+    o.order_state as order_status,

Built With

  • datahub
Share this project:

Updates