Cross-Domain Break Predictor

Warns you on the PR, before merge, when a schema change breaks something downstream using DataHub's real lineage graph, not guesswork. Writes the review back into DataHub too.

Comment

Inspiration

Data pipeline changes break things silently. Someone edits a table, and three dashboards away, something quietly stops working nobody finds out until a report looks wrong days later. DataHub's own hackathon framing describes this almost exactly: agents that generate or approve pipeline code without knowing what's actually connected to what. I wanted to close that gap at the one moment it actually matters before the change merges, not after it breaks something.

What it does

Cross-Domain Break Predictor is a GitHub Action that triggers when a pull request changes a dbt model. It asks DataHub's MCP Server what actually depends on that table downstream, then posts a plain-English warning directly on the PR naming the real affected tables and their owners. If nothing depends on the changed table, it says so, proven safe rather than silent. It then writes a break-predictor-reviewed tag back into DataHub itself, so anyone browsing the catalog inherits that context too, not just PR readers.

How i built it

I self-hosted DataHub on an AWS EC2 instance (after a genuinely long detour through Oracle Cloud capacity limits and payment verification issues on several other providers) and ingested real lineage from jaffle_shop_duckdb dbt Labs' own canonical sample project via dbt-duckdb, rather than using DataHub's pre-built sample dataset. That choice mattered. It meant every lineage result in this project is genuinely derived from real dbt models, not hand-faked or pre-seeded for the demo. The agent itself is a Python script using fastmcp to call DataHub's MCP Server (search, get_lineage, add_tags), triggered by a GitHub Actions workflow on every pull request.

Challenges i ran into

Nearly every layer of this stack changed shape while we were building against it. mcp-server-datahub's tool parameter names differed between versions we tested (direction vs. upstream, hops vs. max_hops), mutation tools weren't available at all in the version our first environment could install (capped at 0.4.0 by a Python 3.10 constraint i didn't initially understand), and add_tags itself failed the first time because DataHub doesn't auto-create a tag entity on first use it has to exist first. Separately, self-hosting DataHub on a memory-constrained instance caused a real OpenSearch OOM crash during testing, which i fixed with a memory cap and a restart policy rather than just hoping it wouldn't happen again. None of these were edge cases i anticipated going in they were things i only found by actually running the full stack against real infrastructure.

Accomplishments that i am proud of

Everything in this submission is independently verifiable, not staged. Two real, distinct pull requests demonstrate different correct outcomes (a real warning with named dependents, and a genuine low-risk clearance for a table with no dependents) proving the agent is actually evaluating risk, not just always warning. A third PR demonstrates the full write-back loop: a comment on GitHub and a tag change in DataHub from the same triggering event. All of it runs against a real, live, self-hosted DataHub instance a judge can trigger themselves by opening a PR against the repo's duckdb branch.

What i learned

Documentation describing an SDK or MCP server's capabilities can lag or diverge from what's actually installable in a given environment several times, the fix was to inspect the running server's own tool schema directly rather than trust what the docs said. I also learned that authenticity in a demo is worth real infrastructure effort: using a recognizable, real dbt project instead of a canned sample dataset took days longer, but it's the difference between a submission a skeptical judge can verify and one they have to take on faith.

What's next for Cross-Domain Break Predictor

Extending the fan-out detection to multi-hop chains beyond the immediate downstream layer, adding column-level (not just table-level) impact analysis using the lineage paths DataHub's MCP Server already exposes, and exploring whether the same pattern could flag risk before generation feeding lineage context to a code-generation agent so it avoids the breaking change in the first place, rather than only catching it after the fact.

Built With

Share this project:

Updates