Blast Radius - Catch schema changes before they surprise you

Inspiration

Anyone who has worked in data long enough has lived this nightmare: someone renames a column or changes a column type from string to int, and three weeks later a production ML model is making garbage predictions. By the time anyone notices, the damage is done. A broken recommendation engine, an incorrect risk score, or a failed customer pipeline because a feature pipeline silently failed.

Current tools notify you when a data pipeline crashes, but they're blind to silent schema drift that corrupts downstream ML models. I built BlastRadius to catch these silent breaks before they cost real money.

What it does

BlastRadius monitors datasets for schema changes (whether a field is added, removed, or has its data type modified). When a change is detected, it does far more than simply log an event. It walks DataHub's lineage graph to identify every downstream feature table, model, and deployment, regardless of how deep or complex the graph branches.

In the demo environment, a single shared feature table feeds two models covering different points in a customer's lifecycle: one predicting who's about to convert, one predicting who's about to churn. When an upstream schema change occurs, BlastRadius traces the fan-out and correctly captures every affected asset across all branches.

Once the blast radius is established, the agent takes three immediate, concrete actions:

• Mutates Metadata in DataHub: Updates the status tag of affected models and deployments (STABLE, DEGRADED, or UNSTABLE) and attaches a clear explanation detailing the exact field-level changes. Anyone inspecting the model in DataHub sees the impact immediately.
• Raises a Native DataHub Incident: Opens a formal DataHub Incident on the source dataset that lists every impacted downstream model and deployment, integrating directly into existing data governance workflows.
• Fires Actionable Slack Alerts: Sends real-time Slack notifications with field-level details so response teams receive immediate, context-rich alerts.

Rather than relying on manual summaries, BlastRadius reads DataHub's version history, computes the diff between schema versions, and calculates downstream impact autonomously. It transforms standard lineage visualization into proactive, automated system protection: "Here is what just broke, here is everything it affects, and here is why."

How I built it

I built BlastRadius directly on DataHub's core platform, which does not include MCP Server, Agent Context Kit, or Skills involved. Specifically:

• BigQuery ingestion connector - To bring real GA sample metadata into DataHub
• Python SDK - to emit my synthetic ML lineage layer (feature table -> two lifecycle-stage models -> their deployments)
• REST API's schema aspect version history - To detect real schema changes (DataHub retains prior schema versions even though its own UI doesn't expose a way to browse the. I identified this gap and built my own detection around it
• GraphQL API - Both to walk the lineage graph via relationship queries and to raise real DataHub Incidents on the affected dataset
• DataHub CLI -Used throughout for verification and demo tooling.

Challenges I ran into

DataHub retains prior schema versions, but its UI doesn't expose a way to browse them. I found this by directly querying the aspect API and built the detection logic around it.

The trickiest bug was a relationship-query direction that returned zero downstream results despite the lineage being real. I resolved it by empirically testing every direction/parameter combination against the live graph instead of trusting documentation, which turned out to be inconsistent on this point.

I also discovered that mlModel and mlModelDeployment entities aren't valid targets for DataHub's Incidents feature, only dataset is, in this version. I adapted by raising the incident on the source dataset and naming the affected models in its description, then filed this as a GitHub issue upstream.

Finally, DataHub's search and lineage-graph UI don't reliably surface ML entity types, so I verify results via the CLI and API throughout rather than relying on the UI.

Accomplishments that we're proud of

• A lineage walk that genuinely branches. One schema change correctly finds two independent models and both their deployments, not just a single chain.

• Real detection, not simulated: I diff actual DataHub schema version history, not a scripted "fake" event.

• A working DataHub Incident, raised natively and correctly scoped once I learned which entity types actually support it.

• Three schema-change types: additions, removals, type changes, each classified by severity and reported with the exact fields involved.

I found and reported a real gap in DataHub itself.

What I've learned

• DataHub's documentation and its actual API behavior don't always agree; The only way to know for sure was to test empirically against a live instance, and not trust what the docs implied.

• A platform's UI maturity and its API/backend completeness are two different things. DataHub's ML entity types are fully supported under the hood but thin in the frontend. I had to verify almost everything through the CLI and direct queries instead of trusting what I saw on screen.

• "Generic" code only proves itself once you test it on something that isn't the easy case. I didn't fully trust my own lineage walk until I built a second model specifically to try to break it.

• Reset scripts and demo tooling need the same rigor as the actual product. A "reset" that doesn't fully reset caused more confusion during my own testing than any real bug did.

What's next for Blast Radius

Right now, BlastRadius runs on demand; the natural next step is converting it into a continuous listener via DataHub's Actions framework to react to schema changes in real time.

I would also like to go beyond the two synthetic models in this demo and test the lineage walk against a real, messier production graph, which would have more hops, more branches, more entity types.

Built With

• datahubcli
• google-bigquery
• graphql
• python
• restapi