Inspiration
A schema change can look completely harmless inside a pull request while being catastrophic several steps downstream.
A data engineer might drop a column from an upstream dbt model without knowing that the table ultimately feeds a production ML model. The PR passes review, gets deployed, and the resulting failure may not be discovered until a feature pipeline or model starts behaving incorrectly.
The problem is not simply a lack of lineage. The problem is that lineage and software development workflows are disconnected.
We built ContextGuard to bring DataHub's organizational context directly into the GitHub PR where the decision is being made. Instead of asking an engineer to manually investigate downstream dependencies, ContextGuard automatically determines the blast radius of a schema change and tells them whether an active ML consumer is actually at risk.
What it does
ContextGuard is an autonomous CI/CD guardrail powered by DataHub.
When a pull request modifies a tracked SQL/dbt schema, ContextGuard:
- Resolves the changed dataset using DataHub metadata.
- Traverses DataHub's lineage graph to identify downstream
MLFeatureTableandMLModelentities. - Classifies the schema change as breaking or safe.
- Blocks the PR only when a breaking change has an active downstream ML consumer.
- Explains the blast radius directly in the GitHub PR.
- Uses a CodeGen Agent to suggest a backwards-compatible fix grounded in the actual DataHub schema and lineage context.
- Writes a scoped risk marker back to DataHub so the risk becomes visible to other users of the affected model.
- Automatically cleans up the risk marker when the PR is resolved.
The system deliberately follows a "suggest, never blindly commit" model. Repository and metadata mutations are narrowly scoped, auditable, and either require human approval or use a controlled ownership-based fast path.
How we built it
The core architecture is a GitHub Action orchestrating several specialized agents around DataHub's MCP Server.
GitHub → ContextGuard → DataHub → GitHub
The interceptor detects PRs affecting tracked SQL/dbt paths and maps the changed file to a DataHub dataset.
The Blast Radius Agent uses DataHub context to resolve entities, traverse downstream lineage, inspect metadata, compare schemas, and examine dataset queries. This lets the system reason about actual downstream dependencies rather than relying only on the contents of the PR diff.
A Change Classifier then distinguishes dangerous changes such as dropped columns and narrowed types from safe changes such as nullable column additions and type widening. Only a breaking change with an active downstream ML model triggers a block.
For breaking changes, the CodeGen Agent generates a backwards-compatible patch using the real schema and lineage context. Depending on the situation, it can produce a native GitHub suggested change or create a companion fix branch/PR when a new file is required.
Finally, the Governance Agent writes a scoped risk marker or governed lifecycle proposal back to DataHub. These changes are automatically resolved when the originating PR is merged or closed.
We also built local simulation tooling and a genuine MCP server backed by the same fixtures, allowing the complete workflow to be demonstrated without requiring every user to have a DataHub Cloud account.
Challenges we ran into
The hardest part was deciding when ContextGuard should actually intervene.
A safety tool that blocks every schema change quickly becomes noise, so we had to distinguish genuinely breaking changes from safe evolution. We chose a deliberately conservative initial classifier: dropped columns, narrowed types, and detected rename-as-drop/add cases are treated as breaking, while nullable additions and type widening are considered safe.
Another challenge was delivering generated fixes through GitHub's review model. GitHub suggested changes work well when the fix modifies existing lines, but they cannot directly create a new file. We therefore implemented two delivery paths: native suggested changes for inline fixes and a companion fix-branch PR for changes requiring new files.
We also had to treat DataHub mutations carefully. Instead of blindly modifying metadata, ContextGuard uses ownership-aware and governed workflows, with scoped tags, proposals, and automatic cleanup.
Finally, we had to account for lineage freshness. If a DataHub instance is only ingested nightly, a dependency created that same day may not yet be visible to ContextGuard.
Accomplishments that we're proud of
We are particularly proud that ContextGuard goes beyond simply reading DataHub metadata.
It creates a complete feedback loop:
Schema change → lineage analysis → risk decision → PR intervention → generated fix → metadata write-back → automatic cleanup
The project demonstrates both sides of DataHub's value for agents: context for making a decision and a shared graph for preserving the result.
We also built two complete fix-delivery paths and generated example outputs from actual pipeline runs rather than manually written demonstrations.
Most importantly, the system is designed around a real engineering workflow: developers continue working in GitHub, while DataHub supplies the organizational and dependency context that GitHub alone cannot provide.
What we learned
We learned that effective AI agents are not necessarily about giving an LLM more autonomy. They are about giving it better context, constrained tools, and clear decision boundaries.
DataHub's lineage graph turns an otherwise ambiguous schema diff into a concrete dependency problem. Instead of asking an agent, "Could this change break something?", we can give it the actual downstream entities, metadata, ownership, schema, and usage context needed to make a much more useful decision.
We also learned that autonomous systems need strong governance boundaries. ContextGuard therefore separates analysis, classification, suggestion, reporting, and mutation, with narrow permissions and human approval where appropriate.
What's next for ContextGuard
The next major improvement is better rename detection. Currently, a column rename appears as a drop plus an add in the raw schema diff, so ContextGuard conservatively treats it as breaking. We want to combine query-pattern analysis with commit context to distinguish genuine drops from intentional renames.
Other planned improvements include:
- Configurable severity thresholds for different teams and domains.
- Slack notifications alongside GitHub reporting.
- Deeper production ML protection using DataHub's ML lineage.
- Additional governance workflows.
- An upstream contribution to DataHub's MCP ecosystem: a
classify_schema_diffhelper tool.
Our longer-term goal is for ContextGuard to become a general context-aware safety layer for data and ML engineering, where agents can understand not only what changed, but who and what depends on it before allowing that change to reach production.
Log in or sign up for Devpost to join the conversation.