Cascade

Inspiration

Schema changes break things silently. A column type gets narrowed, and a revenue dashboard starts understating every number — but nothing errors, nothing alerts. We built Cascade because loud failures are a solved problem. It's the quiet ones that cost you.

What it does

Cascade is an autonomous agent that predicts which downstream consumers will silently produce wrong results after a proposed schema change. It reads DataHub's lineage graph through DataHub's own MCP Server, analyzes the actual SQL and DAX behind each consumer to distinguish a SUM(col) from a SELECT col, estimates how long wrong data would go unnoticed, and writes governance warnings back into DataHub so the next engineer inherits the knowledge.

How we built it

• DataHub MCP Server driven in-process via FastMCP's in-memory transport — real MCP protocol, no subprocess
• Python async architecture with Pydantic models, multi-provider LLM support (Groq, Gemini, OpenAI, Anthropic, Ollama), and a deterministic rule-based fallback
• Three interfaces: web chat UI, terminal REPL, and an MCP server exposing 5 tools
• 97 tests including property-based tests with hypothesis
• Tested against DataHub Core v1.5.0 with the showcase-ecommerce datapack (34 downstream consumers across Snowflake, dbt, Looker, PowerBI, and Tableau)

Challenges we ran into

• DataHub's MCP Server doesn't expose transformation logic (viewProperties.logic) or column-level lineage as a batch operation — we had to build a hybrid MCP + GraphQL reader and document why
• Free-tier LLM rate limits mean the first 12 consumers get LLM reasoning and the rest fall back to rules — we made this graceful rather than failing
• The DataHub UI renders documentation from the primary sibling only, so writes to just the Snowflake URN were invisible — had to discover and write to every sibling

What we learned

• The difference between "what's connected" and "what will actually break" is decidable from metadata DataHub already holds
• Writing results back into the metadata graph — not just printing them — is what makes an agent genuinely useful to a team
• DataHub's MCP Server is powerful for reads but has gaps for agents that need transformation logic and column-level provenance

Open-source contributions

• datahub-project/datahub-skills #121 — Cascade skill added to the DataHub Skills Registry
• acryldata/mcp-server-datahub #206 — MCP tool surface gaps documented upstream

Built With

• datahub-mcp-server
• fastapi
• fastmcp
• groq
• httpx
• mcp-server-datahub
• pydantic
• pytest
• python