ContextSeal

Every data change ships with proof, not confidence. ContextSeal uses DataHub context to block risky schema changes and generate safe, reviewable migration packages.

Comment

Inspiration

AI coding agents can generate valid SQL and dbt code while missing the organizational context that makes a data change safe. A repository alone does not reveal that a field feeds a dashboard several hops away, carries sensitive-data governance signals, powers another pipeline or model, appears in observed queries, or belongs to another team.

DataHub already contains that missing context: schemas, lineage, ownership, governance, quality signals, incidents, and query context. But metadata alone is not enough. A reviewer still needs to know whether a proposed change is safe, what will be affected, what should be generated instead, who must approve it, and what evidence should remain after the decision.

We built ContextSeal to turn DataHub context into an enforceable pre-merge certification boundary instead of leaving it as passive metadata.

The core idea is simple: a repository can see code; DataHub can see what that code may break. ContextSeal turns that context into a deterministic decision, a safer alternative, and durable proof.

What it does

ContextSeal is a DataHub-native certification agent for risky schema changes.

A developer proposes a column rename, drop, or type change. Before the destructive change can move forward, ContextSeal evaluates the available DataHub context, reconstructs downstream impact paths, and applies a deterministic risk policy.

In the public judge demo, a direct rename from customer_email to contact_email receives a deterministic risk score of 80 and is BLOCKED.

The fixture-backed impact view exposes five downstream consumers across data jobs, datasets, dashboards, and an ML model. ContextSeal preserves the dependency path and hop count behind that impact so the blast radius is explainable rather than guessed.

The deterministic policy findings include:

  • breaking lineage
  • sensitive-data governance
  • query-usage evidence

Instead of allowing the destructive rename, ContextSeal converts the request into an expand-migrate-contract strategy and generates five reviewable files:

  • a dbt migration model
  • schema tests
  • a rename parity data test
  • rollback SQL
  • an impacted-owner brief

The generated files are tied back to a grounding manifest, allowing a reviewer to inspect not only what was generated but also why it was generated.

The original destructive rename is never what the human reviewer approves. The reviewer can approve only the generated safe migration scope.

That scoped approval unlocks a SHA-256 change passport that binds the request, deterministic evidence, generated artifacts, approval scope, migration strategy, and validity window into one durable decision record.

ContextSeal also includes an optional local AI explanation layer powered by Ollama. AI runs only after the deterministic verdict has already been fixed. It can generate operator guidance, migration rationale, reviewer notes, and owner-facing explanations, but it cannot change the risk score, evidence states, or approval authority.

The public GitHub Pages demo is deliberately fixture-backed and does not modify a live DataHub instance.

A separate recorded disposable-local DataHub proof demonstrates the real MCP integration using synthetic metadata. That proof records ten bounded MCP reads, six downstream assets across DATASET, DATA_JOB, and DASHBOARD entity types, three write-back operations applied on the first run, the same three operations skipped on an idempotent retry, and durable read-back verification.

The final public demo is approximately 2 minutes and 5 seconds and walks through the complete story from blocked request to grounded impact, deterministic verdict, generated package, scoped human approval, change passport, and recorded DataHub write-back proof.

How we built it

ContextSeal combines a deterministic application core with DataHub context, bounded agent behavior, generated engineering artifacts, and explicit evidence states.

The main components include:

  • a deterministic Node.js core
  • typed change and approval contracts
  • DataHub OSS / Core Platform
  • the official DataHub MCP Server
  • support for local stdio MCP transport and DataHub Cloud streamable HTTP transport
  • bounded multi-hop lineage traversal
  • a versioned deterministic risk policy
  • a dbt artifact generator
  • human scope-bound approval
  • SHA-256 passport generation
  • structured-property, description, and document write-back
  • verify-then-skip idempotency
  • an optional local Ollama adapter
  • a deterministic artifact conformance sandbox
  • local dbt execution proof using dbt and DuckDB
  • Docker
  • GitHub Actions
  • a dependency-free responsive judge dashboard
  • a reusable DataHub schema-change certification skill

The DataHub MCP read path uses:

  • get_entities
  • list_schema_fields
  • get_lineage
  • get_lineage_paths_between
  • get_dataset_queries

The approved write-back path uses:

  • add_structured_properties
  • update_description
  • save_document

ContextSeal deliberately separates reading context from mutation authority. Live mutations are gated and disabled unless the required conditions are explicitly satisfied.

The generated migration package is also checked by a deterministic local conformance sandbox against its manifest, hashes, and grounding contract.

A separate dbt proof executes the generated bundle locally using dbt and DuckDB. This verifies executable artifact behavior without pretending that production warehouse SQL was executed.

The optional GitHub delivery path similarly defaults to a token-free reviewer-ready PR bundle and dry-run request validation. An actual draft PR call is explicit and token-gated rather than silently executed.

Challenges we ran into

The hardest design problem was separating an unsafe original request from a safe generated alternative.

A direct breaking rename can correctly receive a BLOCKED verdict while the generated staged migration remains eligible for scoped human approval. Treating those as the same decision would either allow a destructive operation or make useful automation impossible.

We also had to preserve strict boundaries between:

  • fixture evidence
  • recorded local evidence
  • live operations
  • unavailable services
  • stale evidence
  • operations that were never run

Those states must never collapse into a generic success claim.

One important MCP integration lesson came from a tool-level validation failure returned through the MCP transport with an isError signal. The transport itself had successfully delivered the response, but the business operation had failed.

A client that checked only whether an MCP response arrived could mistakenly record that as successful evidence.

We changed ContextSeal to fail closed whenever that tool-level error state is present and added a regression test before accepting the operation as PASS.

Write-back created another challenge: idempotency.

Certification metadata should not be duplicated every time an operator retries the same passport. ContextSeal therefore verifies existing state before mutation and uses a verify-then-skip strategy.

In the recorded disposable-local DataHub proof, the first run produced three APPLIED mutation receipts. Repeating the same operation with the same passport produced three SKIPPED receipts instead of duplicate metadata.

We also had to make the public demo reproducible for every judge without pretending that GitHub Pages was connected to a live DataHub tenant. The solution was to keep the public graph experience explicitly fixture-backed while presenting the separate recorded DataHub MCP evidence with equally explicit local and synthetic boundaries.

Accomplishments that we're proud of

We are especially proud that ContextSeal does more than display metadata or generate an explanation.

It changes what the system is allowed to do.

Our main accomplishments include:

  • deterministic risk findings that AI-generated text cannot overwrite
  • explainable multi-hop downstream paths rather than only a flat asset count
  • a destructive schema rename blocked before merge
  • a safe expand-migrate-contract alternative generated automatically
  • five inspectable migration artifacts grounded in captured context
  • manifest-linked generated outputs
  • a rename parity test rather than relying only on generated SQL
  • rollback SQL included in the review package
  • an impacted-owner brief that connects technical change to organizational impact
  • human approval restricted to the generated safe scope
  • a SHA-256 change passport binding request, context, evidence, artifacts, and approval
  • explicit PASS, NOT_RUN, FIXTURE, and other evidence states
  • a bounded local AI explanation layer that remains non-authoritative
  • a deterministic local artifact conformance sandbox
  • a real local dbt execution proof
  • a reviewer-ready PR bundle
  • token-free draft-PR request validation
  • fail-closed DataHub mutation gates
  • idempotent DataHub write-back behavior
  • durable DataHub read-back verification
  • a recorded disposable-local DataHub proof with ten MCP reads
  • six recorded downstream assets across DATASET, DATA_JOB, and DASHBOARD
  • three APPLIED write-back operations
  • three SKIPPED verify-then-skip retry operations
  • structured properties, description context, and a decision document written to synthetic DataHub metadata
  • a reusable DataHub change-certification skill

We also submitted the reusable datahub-schema-change-certification workflow upstream to the DataHub Skills project as pull request #35.

At submission time that contribution is OPEN / NOT MERGED / AWAITING MAINTAINER REVIEW. We do not claim or imply maintainer approval.

What we learned

The biggest lesson was that context is most valuable when it changes an action, not when it only improves an answer.

DataHub can provide much more than background information for an LLM. Its schemas, lineage, ownership, governance signals, and usage context can become part of an enforceable engineering decision boundary.

We also learned that deterministic evidence and AI explanation should have different authority.

AI is useful for turning grounded evidence into understandable guidance, but it should not be allowed to rewrite the facts that determine whether a change is safe.

Another lesson was that honest evidence states make an autonomous system more credible, not less impressive.

FIXTURE should remain fixture.

NOT_RUN should remain not run.

Recorded local proof should not be presented as hosted live execution.

Synthetic metadata should not be presented as production evidence.

A successful protocol response should not automatically be interpreted as a successful business operation.

Finally, durable write-back matters. If a certification result exists only in a chat session, the next engineer or agent starts again from zero. Writing bounded decision context back to DataHub allows the next operator to inherit the decision and its evidence.

What's next for ContextSeal

Next we want to extend ContextSeal in several directions:

  • respond to maintainer feedback on the DataHub Skills contribution
  • support target-derived normalization for more DataHub entity types
  • add signed reviewer identities
  • add replay protection for approvals and passports
  • exercise the optional token-gated draft PR path against a real GitHub branch after explicit approval
  • add warehouse-specific sandbox executors
  • extend certification beyond column changes
  • support dbt model changes
  • support pipeline and schedule changes
  • expand durable decision inheritance across more metadata objects
  • strengthen reusable policy packs for different organizational environments

The longer-term goal is to make metadata context part of the engineering control plane for autonomous coding and data agents: not just something an agent can read, but something that can safely constrain, explain, and certify what the agent is allowed to change.

Honest boundaries

ContextSeal is a hackathon prototype.

The public GitHub Pages experience is fixture-backed. It does not perform hosted live DataHub mutations.

The hosted page also does not perform live Ollama inference. It replays separately recorded local AI proof.

The recorded DataHub integration proof comes from a disposable-local DataHub environment using synthetic metadata. It is not production or customer data.

The recorded write-back proof demonstrates bounded structured-property, description, and document mutations plus durable read-back. It does not claim production deployment.

The local dbt proof demonstrates isolated executable bundle behavior using DuckDB. It is not production warehouse execution.

ContextSeal does not auto-merge production changes.

It does not claim measured customer impact.

It does not claim a security or compliance guarantee.

The upstream DataHub Skills pull request is open and not merged.

Only operations backed by named evidence are presented as PASS.

Built With

Share this project:

Updates