ChangeGuard

Inspiration

Metadata changes rarely fail where they are made. A removed field can silently break a transformation, dashboard, model, or AI agent several lineage hops downstream. Teams often discover the damage only after deployment because the blast radius and required verification were never locked in advance.

What it does

ChangeGuard is an evidence-first safety agent for DataHub metadata changes. It reads real catalog assets and direct lineage through DataHub MCP, compares before and proposed-after metadata, propagates breaking changes across multi-hop lineage, locks asset-specific checks before results are observed, and returns a deterministic keep, pause, or rollback recommendation.

The demo proposes removing payment_method_code from an orders dataset. The proposal is never written to DataHub. ChangeGuard detects the breaking change, follows three real lineage hops, identifies 17 impacted assets, locks 17 checks, and recommends pause. It can then write a hidden UNPUBLISHED evidence document back to DataHub only through an explicit three-gate approval flow.

How we built it

• DataHub OSS v1.6.0 quickstart
• official mcp-server-datahub over stdio
• DataHub search, entity, lineage, and document tools
• deterministic Python domain core with no LLM in the decision path
• append-only JSONL evidence ledger with file locking and fsync
• dependency-free local judge application plus a public hosted demo

The adapter walks one lineage hop at a time, preserving real direct edges instead of guessing a graph from flattened multi-hop results.

Challenges

The official quickstart needed two reproducible compatibility fixes: DataHub GMS was mapped to port 18080 because port 8080 was already occupied, and the generated v1.6.0 compose setup required an explicit datahub-gms network alias on this Docker runtime.

Accomplishments

• 17 supported assets and 16 direct edges in the verified live snapshot
• 17 impacted assets across hops 0–3
• multi-hop impact recall of 100% versus 69.23% for a direct-only baseline across five deterministic cases
• hidden UNPUBLISHED evidence writeback verified across 17 related assets
• 35 automated tests passing and release audit passing
• public source, public judge demo, and 4K hard-captioned demo video
• upstream DataHub MCP documentation PR: https://github.com/acryldata/mcp-server-datahub/pull/144

What we learned

Prediction and verification must be separate phases. If an agent can revise its predicted blast radius after seeing test results, the evidence loop becomes biased. ChangeGuard fingerprints the prediction, impacts, and required checks first; later results may change only the final recommendation.

What's next

• CI adapters for contract, pipeline, model, and query checks
• distributed evidence-ledger support for multiple runners
• broader real-world benchmark suites and failure injection
• policy packs for common warehouse and AI-agent change patterns

Competition-period disclosure

All DataHub-specific application code was newly created during the competition period. General reliability ideas and existing runtime libraries influenced the design, but no pre-existing application was submitted as the project.

Built With

• datahub