ChangeSafe

Inspiration

Schema migrations look deceptively small in a pull request. A dropped column may actually sit upstream of a certified executive dashboard, an operational sync, and a production ML feature. Code review rarely exposes that complete organizational context. We wanted an agent that turns DataHub's context graph into a concrete, auditable safety decision before deployment.

What it does

ChangeSafe accepts a proposed dataset change and verifies the target field against DataHub. It traces three hops of downstream lineage, collects owners and governance signals, and calculates an explainable risk score. The agent returns an ALLOW, REVIEW, or BLOCK decision, identifies the exact blockers, generates a staged migration plan, and writes the decision record back to DataHub for future humans and agents.

In the included scenario, dropping customer_email receives a CRITICAL 83/100 score. ChangeSafe finds six impacted assets across dbt, Airflow, Snowflake, Looker, an ML model, and a data product; it blocks the deployment and identifies six owner teams to notify.

How we built it

The core is a dependency-free Python policy engine behind a small HTTP application. A production MCPGateway connects to DataHub's official Streamable HTTP MCP endpoint and calls get_entities, get_lineage, and save_document. A fixture gateway implements the same interface so judges can run the full workflow without credentials. The browser UI visualizes the decision, score, ownership, and multi-hop evidence.

The risk model is deliberately deterministic and explainable. It combines operation severity, direct consumer types, transitive reach, PII and governance tags, non-null constraints, and certified downstream assets. This means AI can assist with intent, but it cannot hallucinate lineage or silently override the safety gate.

Challenges we ran into

The hardest design question was how to make an autonomous agent safe enough for a deployment gate. A purely generative decision would be difficult to audit. We separated DataHub evidence collection from a deterministic policy layer, preserved every scoring input in the response, and made unknown fields fail closed. We also isolated the transport behind a gateway so the same tested workflow runs against a local fixture or a real MCP endpoint.

Accomplishments that we're proud of

• Meaningful use of DataHub schema, tags, certification, owners, and three-hop lineage.
• A closed-loop workflow that writes its decision record back into DataHub.
• A zero-setup demo with no third-party runtime dependencies.
• An inspectable score rather than a black-box AI verdict.
• Automated unit and browser tests covering the end-to-end path.

What we learned

DataHub's value for agents is not just search. The context graph lets an agent reason across technical dependencies and organizational responsibility, while mutation tools let it leave durable knowledge behind. We also learned that safe automation benefits from combining rich graph context with deterministic decision rules.

What's next for ChangeSafe

Next we would add a GitHub check that parses SQL and dbt diffs automatically, column-level lineage paths, policy profiles per domain, owner acknowledgements, and a post-deployment monitor that closes the DataHub decision record after the deprecated field has zero consumers.

Built With

• context
• css
• datahub
• github
• html
• javascript
• mcp
• model
• protocol
• python
• server