DataGuard Ai

Inspiration Data teams face a constant nightmare: an upstream developer modifies or renames a database column in a microservice, and downstream dbt models, Airflow DAGs, and ML feature stores silently crash in production. Finding what broke, tracing the dependencies across complex pipelines, and creating hotfix PRs often takes hours or days. We built DataGuard AI to eliminate silent pipeline breaks entirely.

Inspired by our desire to make AI agents truly context-aware, we asked a fundamental question:

What if an AI agent used DataHub’s context graph to automatically detect breaking schema changes, heal downstream code before it crashes, and sync state back to the entire organization? What it does DataGuard AI is an autonomous, context-aware data stack protector that acts as a bridge between CI/CD developer workflows and DataHub's metadata graph.

Listens to Schema Changes: Intercepts upstream Pull Requests (or webhooks) containing database schema modifications.

Performs Impact Analysis via MCP: Connects to DataHub’s MCP Server and Agent Context Kit to traverse multi-hop lineage (from raw PostgreSQL tables down to dbt models, Airflow DAGs, and ML models).

Self-Heals Downstream Code: Generates production-ready Git PRs that refactor affected SQL queries and Python scripts, complete with explanatory comments.

Bi-Directional Graph Write-Back: Updates DataHub in real time by applying 🏷️ schema-breaking-change tags, raising official DataHub Incidents, and proposing updated Glossary terms.

How we built it Frontend: Next.js (App Router), TypeScript, and Tailwind CSS configured in a clean, light-themed visual system with interactive node lineage graphs powered by @xyflow/react.

Agent Context Layer: Powered by @modelcontextprotocol/sdk and datahub-agent-context to execute multi-hop lineage traversal (get_lineage), field inspection (list_schema_fields), and write-back calls (add_tags, create_incident).

Multi-Agent Orchestrator: Built with a three-stage pipeline (Impact Agent $\rightarrow$ Healing Agent $\rightarrow$ Governance Agent) that parses schema diffs, refactors code syntax safely, and commits hotfix PRs to downstream repositories.

Open-Source Contribution: Authored a standalone DataHub Skill recipe located in /contrib/datahub-pr-impact-skill/ that enables any MCP-connected AI agent to map raw Git PR diffs directly to DataHub URNs.

Challenges we ran into Bi-Directional Graph Syncing: Moving beyond simply reading DataHub metadata to reliably writing back tags and raising incidents required fine-tuning our MCP tool schemas to handle complex URN formats correctly.

Accurate Lineage Traversal: Parsing multi-hop dependencies across different data stack tools (Postgres $\rightarrow$ dbt $\rightarrow$ Airflow) meant handling heterogenous entity types gracefully without losing track of column-level mappings.

Generating Non-Breaking Code Fixes: Ensuring the Healing Agent refactored downstream SQL and Python files without altering business logic required strict AST-aware prompt constraints and context injections from DataHub.

Accomplishments that we're proud of True Self-Healing Loop: Successfully built a working, bi-directional feedback loop where upstream code changes automatically fix downstream repositories and write audit events back into DataHub.

Light-Themed & Approachable UX: Designed an intuitive, approachable platform interface that displays complex multi-hop lineage and AI decision reasoning with complete transparency.

Meaningful Open-Source Contribution: Contributed the datahub-pr-impact-skill recipe to expand DataHub's open-source ecosystem for future developer agents.

One-Command Evaluation: Built a seamless testing suite (npm run demo) that executes the entire pipeline end-to-end against a local DataHub Docker instance.

What we learned Metadata is the fuel that elevates AI agents from novelty helpers to reliable enterprise tools; without DataHub's complete context graph, autonomous code refactoring would hallucinate or break dependencies. Write-backs are essential for team trust—letting agents contribute findings back to the context graph ensures both humans and other agents inherit fresh governance metadata.

What's next for DataGuard AI Native CI/CD Integrations: Package DataGuard AI into a turnkey GitHub Action and GitLab Pipeline plugin for one-click repository installation. Automated Shadow Testing: Run generated fix PRs against isolated staging database instances using DataHub data quality assertions before opening PRs.

Slack & Teams Incident Bot: Expand notification channels so data teams receive real-time interactive Slack alerts with quick "Approve Fix" buttons directly linked to generated PRs.

Built With

• datahub
• react