DataGuardian — Autonomous Data & ML Reliability Agent

💡 The Inspiration

The idea came from a simple question:

What if an AI agent could understand an organization's entire data ecosystem before taking action?

Consider a seemingly harmless change:

customers.email → removed

In a traditional workflow, someone may discover the consequences only after a downstream pipeline fails.

With DataGuardian, the agent can trace the dependency chain:

customers.email
      ↓
customer_features.email_domain
      ↓
fraud_detection_model
      ↓
fraud_prediction_api

Instead of waiting for something to break, DataGuardian identifies the potential blast radius, calculates the risk, explains why the change matters, and proposes a remediation.

This led us to build around a simple principle:

Give AI agents context before giving them autonomy.

What it does

Data Guardian is designed as a multi-agent system with specialized responsibilities:

🔎 Data Agent

Uses DataHub context to investigate:

datasets
schemas
columns
lineage
downstream dependencies
ownership

🛡️ ML Guardian

Analyzes whether a data change can affect:

features
training datasets
ML models
production deployments
prediction services

💻 Code Agent

Generates metadata-aware artifacts such as:

SQL
dbt transformations
Airflow tasks
validation tests
migration code

🔧 Remediation Agent

Turns the analysis into an actionable response, including:

recommended fixes
incident reports
validation steps
code patches
remediation plans

An orchestrator coordinates these agents so that the system behaves more like a data engineering team than a simple chatbot.

How we built it

DataGuardian is built around a modular architecture.

                 ┌──────────────────┐
                 │   Web Dashboard   │
                 └────────┬─────────┘
                          │
                          ▼
                 ┌──────────────────┐
                 │ Agent Orchestrator│
                 └────────┬─────────┘
                          │
             ┌────────────┼────────────┐
             ▼            ▼            ▼
        Data Agent   ML Guardian   Code Agent
             │            │            │
             └────────────┼────────────┘
                          ▼
                    DataHub Context
                          │
                          ▼
                 Lineage + Metadata
                          │
                          ▼
                 Risk & Impact Engine
                          │
                          ▼
                    Remediation

Technology Stack

Frontend

React
Vite
Tailwind CSS
GSAP
React Flow
Lucide icons

Backend

Python
FastAPI

LLM-powered agents
Tool-based agent architecture
Structured outputs

Data Context

DataHub
DataHub APIs / MCP integration
Dataset and column lineage

Infrastructure

Docker
Git
REST APIs

🚨 The Main Demo Scenario

Our primary scenario demonstrates what happens when a critical data field changes.

Suppose:

customers.email

is removed.

DataGuardian investigates the change and discovers that the column contributes to a downstream feature:

customers.email
       ↓
customer_features.email_domain
       ↓
fraud_detection_model
       ↓
fraud_prediction_api

The system then evaluates the blast radius.

For example:

🚨 CRITICAL DATA INCIDENT

Affected datasets:      1
Affected features:      1
Affected pipelines:     1
Affected ML models:     1
Production services:    1

Risk Score: 96/100

Rather than simply reporting the problem, the agents continue working.

The Code Agent can generate a remediation such as:

Restore schema compatibility
Update the feature transformation
Add a schema validation test
Validate downstream model compatibility
Prepare a migration/patch

This demonstrates the difference between an AI assistant that answers questions and an AI agent that can perform meaningful data engineering work.

📊 Explainable Risk Scoring

We intentionally designed the risk system to be explainable rather than relying entirely on an opaque AI-generated score.

A simplified model can be represented as:

$$R = B + \sum_i W_i$$

where:

$R$ = overall risk score
$B$ = baseline risk
$W_i$ = weighted impact of each detected condition

For example, a removed column can carry a higher weight than a newly added optional column, while downstream ML models and production services increase the severity.

This lets DataGuardian explain why an incident received a particular risk score.

The individual component weights can be expressed as:

$$W_i = \alpha_i \cdot I_i \cdot D_i$$

where:

$\alpha_i$ = condition severity coefficient (e.g., column removed = 1.0, column added = 0.3)
$I_i$ = impact multiplier based on downstream count
$D_i$ = depth factor penalizing deeper lineage paths

The final normalized risk score is then:

$$R_{normalized} = \min\left(100, \frac{R - R_{min}}{R_{max} - R_{min}} \times 100\right)$$

The lineage depth penalty decays exponentially with hop distance:

$$D_i = e^{-\lambda \cdot h_i}$$

where $h_i$ is the number of lineage hops from the changed asset, and $\lambda$ is a decay constant (typically $\lambda = 0.3$).

Challenges we ran into

One of our biggest challenges was making the system more than a simple AI chatbot.

A chatbot can answer:

"What might be affected if this table changes?"

But DataGuardian needs to prove the answer using metadata and lineage.

We therefore had to think carefully about:

connecting the agents to DataHub context
representing lineage accurately
tracing dependencies across multiple levels
identifying ML model impact
calculating explainable risk
generating useful remediation code
keeping AI-generated actions grounded in real metadata

Another challenge was designing the system so that it remains useful even when an external LLM is unavailable. We therefore built deterministic components for core analysis and structured the AI layer so that it can be configured with any LLM provider.

What we learned

One of our biggest challenges was making the system more than a simple AI chatbot.

A chatbot can answer:

"What might be affected if this table changes?"

But DataGuardian needs to prove the answer using metadata and lineage.

We therefore had to think carefully about:

connecting the agents to DataHub context
representing lineage accurately
tracing dependencies across multiple levels
identifying ML model impact
calculating explainable risk
generating useful remediation code
keeping AI-generated actions grounded in real metadata

What's next for DataGuardian — Autonomous Data & ML Reliability Agent

DataGuardian is designed as a foundation rather than the end of the idea.

Future versions could:

automatically create GitHub pull requests
integrate with CI/CD pipelines
monitor real-time metadata changes
detect data quality regressions
monitor ML feature drift
automatically validate generated code
connect to Slack or incident-management systems
support multiple DataHub environments
create long-term organizational knowledge from resolved incidents

The long-term goal is to move from:

"AI that understands data."

to:

"AI that safely operates within a data ecosystem."

❤️ Why This Matters

Data engineering teams spend significant time understanding dependencies, tracing failures, investigating schema changes, and determining who or what might be affected.

DataGuardian explores how much of that work can be accelerated when AI agents have access to reliable organizational context.

DataHub provides the context. AI provides the reasoning. DataGuardian connects the two to turn context into action.

That's the idea behind DataGuardian.