CASCADE — Blast Radius Intelligence Agent

The first AI agent that quantifies the financial blast radius of schema changes using DataHub lineage — before anyone hits merge.

Comment

Inspiration

Every data engineering team faces the same silent disaster. A senior engineer renames a column. The PR looks safe. Three days later, seventeen downstream pipelines are broken, two ML models are serving stale features, and a compliance report filed incorrect figures to regulators. The post-mortem takes two weeks. The cost: tens of thousands of dollars.

DataHub has the complete lineage graph to prevent this. No agent has ever been built to act on it proactively — until CASCADE.

What It Does

CASCADE is a three-agent blast radius intelligence system that:

  1. SCOUT Agent — Traverses DataHub's full downstream lineage graph using the MCP Server get_lineage() tool, identifying every affected dataset, ML model, dashboard, and compliance report
  2. VALUATION Agent — Reads ownership metadata, business domain tags, and revenue criticality labels from the Agent Context Kit to assign a financial cost to each broken asset
  3. PATCH Agent — Auto-patches safe downstream assets with corrected SQL, flags regulatory assets for human review, and writes the full incident report back to DataHub using save_document() and add_tags()

The result: a complete blast radius report with financial quantification, generated in under 60 seconds, before anyone hits merge.

How We Built It

  • Agents: Python + Groq (llama-3.3-70b-versatile)
  • API: FastAPI + Uvicorn
  • DataHub Integration: MCP Server tools (get_lineage, get_entities, add_tags, save_document), Agent Context Kit (build_langchain_tools)
  • Dashboard: Vanilla HTML/CSS/JS served directly from FastAPI
  • Deployment: Render (free tier)
  • Trigger: GitHub PR webhook receiver built in FastAPI

Challenges We Faced

The biggest challenge was infrastructure. Running DataHub OSS locally requires 8GB+ free RAM — a constraint in our development environment in Nigeria. We architected CASCADE's SCOUT agent to connect directly to any DataHub GMS endpoint via the real get_lineage() API, with mock lineage data used during development to validate the full pipeline end-to-end.

The second challenge was designing the financial valuation model. We built a tag-based cost multiplier system: revenue_critical assets carry an $18,000 remediation estimate, regulatory assets carry $35,000, and business_critical assets carry $9,000 — reflecting real-world engineering and legal remediation costs.

What We Learned

  • DataHub's Agent Context Kit makes it remarkably straightforward to build agents that reason over organizational metadata
  • The save_document() mutation tool is the key to making agents leave permanent, auditable context in DataHub
  • Financial quantification is the difference between a data quality alert and an executive-level incident report

Open Source Contribution

We propose a new ChangeImpactSkill for the DataHub Skills library — a structured API that exposes blast radius assessment for schema changes, reusable by any team building agents on DataHub. This skill does not exist today and would immediately benefit every DataHub user running AI agents on their data stack.

Built With

Share this project:

Updates