• 

  Deterministic 92/100 migration gate with DataHub lineage evidence, ownership gaps, and downstream blast radius.

• 

  PR-ready impact report, migration plan, SQL guard, dbt test, Decision record, and owner action handoff.

Inspiration

Schema changes rarely fail at the table being edited. They fail in a dbt model, dashboard, replica, or unowned consumer nobody knew depended on the field. DataHub already has the schema, lineage, platform, and ownership context needed to make that risk visible. LineageGuard turns that context into an explicit migration gate before destructive work begins.

What it does

LineageGuard accepts a proposed field rename, type change, or drop and then:

• reads the source entity and schema plus column- and dataset-level lineage;
• scores every downstream asset with an explainable deterministic policy;
• selects PROCEED_WITH_MONITORING, EXPAND_THEN_CONTRACT, or BLOCK_AND_MIGRATE;
• generates an impact report, compatibility SQL, dbt YAML, migration phases, owner actions, and a machine-readable Decision record; and
• optionally writes that Decision document back through DataHub MCP so the next engineer or agent inherits the evidence.

It never executes generated SQL, changes a schema, invents missing owners, or writes to DataHub unless writeback is explicitly requested.

The browser demo proposes renaming ORDER_DETAILS.customer_id to buyer_id. It finds four downstream consumers across dbt, Looker, Power BI, and Snowflake, including an unowned dashboard whose uncertainty blocks a destructive contract phase. Judges can inspect every reason, artifact, and owner action without credentials.

How we built it

The core is a deterministic TypeScript decision engine behind a CatalogAdapter boundary. DemoCatalogAdapter supplies a clearly labeled fixture for the hosted UI. DataHubMcpCatalogAdapter launches the official DataHub MCP server over stdio JSON-RPC and invokes get_entities, list_schema_fields, and get_lineage. It merges dataset- and column-level evidence so uncertain consumers are not silently dropped.

The generated bundle is dataset-specific rather than template theater: relation names come from the DataHub dataset, unsafe identifiers and cast types fail closed, rename/type/drop operations produce different safe review artifacts, and missing metadata raises risk instead of becoming a safety claim. save_document is the only supported write operation and sits behind both an explicit writeback call and the official server's mutation flags.

The Next.js interface is deployed on Vercel. The public repository includes sample output, a machine-readable Decision record, the exact architecture, reproducible setup, CI, and an Apache-2.0 license.

Reproducible process-level evidence

We did not stop at a mocked adapter. The committed official audit launches the published mcp-server-datahub@0.6.0 package with uvx as a real child process. Against a bounded loopback GMS fixture it completes MCP initialize and tool discovery, then exercises the official server's GetEntity, getRelatedDocuments, and GetEntityLineage paths through LineageGuard's get_entities, list_schema_fields, and two get_lineage calls.

A second process enables mutation-tool discovery only long enough to inspect the official save_document input schema. It never performs the write. That distinction lets anyone reproduce the JSON-RPC framing, tool registration, GraphQL requests, response cleaning, and version-gated writeback contract without claiming access to a live DataHub tenant.

The suite has ten passing tests covering deterministic policy behavior, missing-field failure, identifier/type safety, dataset-aware artifact generation, change-specific SQL/dbt behavior, and the exact MCP read/write boundary. GitHub Actions runs type-checking, tests, and the production build.

DataHub challenge fit

LineageGuard combines two tracks:

• Agents That Do Real Work: it turns catalog context into an accountable owner plan and can preserve the result as a DataHub Decision document.
• Metadata-Aware Code Generation & Development: it generates review-ready migration artifacts from actual schemas and lineage before code reaches a pull request.

Challenges we ran into

The hardest integration detail was preserving uncertainty. Column lineage is nested under downstream search results, while some consumers only appear at dataset level. Treating either response as complete would produce a dangerously small blast radius. LineageGuard merges both and increases risk when evidence or ownership is incomplete.

We also discovered that DataHub's root MCP quickstart referenced an npm package that does not exist. We reproduced the failure, verified the maintained uvx distribution, and opened a focused upstream documentation fix at datahub-project/datahub#18892.

Accomplishments

• A polished working application and live CLI integration boundary.
• Deterministic, explainable risk rather than an opaque confidence score.
• Dataset-aware SQL, dbt YAML, migration phases, owner actions, and Decision JSON.
• A reproducible audit against the actual published DataHub MCP process.
• An explicit read/write boundary with fail-closed transport handling.
• Ten passing tests, green CI, a public repository, and a concise 60-second demo.
• A concrete upstream DataHub documentation contribution and response-contract feedback.

What we learned

Missing metadata is not a reason to assume safety; it is evidence that should raise the gate. A useful data agent must preserve that uncertainty, show its reasoning, and leave a durable handoff rather than merely produce plausible text.

What's next

Next steps are a live-tenant contract suite across supported DataHub versions, a pull-request bot that attaches the review bundle to schema diffs, and versioned JSON Schema fixtures for MCP tool results. The writeback boundary will remain explicit: LineageGuard is a control plane for reviewed migrations, not an autonomous production mutator.

Built With

• datahub
• dbt
• devops
• lineage
• mcp
• metadata
• sql
• typescript