Inspiration
A vendor changes what your company is allowed to do with a dataset.
The amendment is one document. The data is already inside an executive dashboard, two machine-learning models, a customer delivery job, and an anonymized derivative. Miss one downstream use, and the company can continue using data in a way the agreement no longer permits - even after the review has been marked complete.
Finding those assets is only half the problem. Someone must interpret exactly what changed, trace every dependency, decide what the new clause means for each use, and leave enough evidence for the accountable owner to act.
Today, governance leads do this manually across contracts, lineage graphs, spreadsheets, tickets, and institutional memory. The work is slow, but incompleteness is the greater danger: one missed asset can preserve the wrong use, while one decision without its clause and rationale must be reconstructed the next time the agreement changes - or the person who made it leaves.
DataHub knows where data goes. Covenant makes it remember what those connections require.
Covenant turns a reviewed agreement change into a cited, differentiated impact plan. It follows live DataHub lineage, determines what the changed terms mean for each downstream use, and records the proposed response and its reasoning back onto every affected DataHub entity.
What it does
Covenant handles an agreement change in three acts.
Recognize
A governance lead uploads a new amendment.
Covenant identifies the governed agreement already registered in DataHub, compares the new document with the prior version, and extracts a cited rule change for human review.
Every proposed rule must point to exact language in the source. An independent deterministic verifier checks those citations, validates the schema, and confirms that the rule type matches the cited language.
If Covenant cannot recognize the agreement or verify the evidence, it stops.
Propagate
Once a human approves the change, Covenant asks DataHub to resolve the governed source and reconstruct its downstream lineage.
The affected set comes from the live DataHub graph - not from a browser-provided list or static fixture.
Covenant applies deterministic governance policy to each downstream asset’s recorded usage classification.
In our canonical case, one amendment produces five different outcomes:
- one executive dashboard remains allowed;
- two machine-learning models require remediation;
- one customer delivery job receives a proposed stop;
- one anonymized derivative remains unresolved for human review;
- one unrelated control asset is verified untouched.
Record
Covenant writes each proposed response back onto the affected DataHub entity as structured properties and tags.
Every response carries its triggering clause, complete lineage path, deterministic policy rule, proposed action, and approval state.
Covenant then reads each entity back to verify that the state landed.
Nothing is automatically stopped, retrained, or executed. A stop remains proposed. A review remains unresolved. Human owners retain authority over consequential decisions.
How we built it
Covenant separates interpretation, verification, propagation, and authority.
The language model handles the upstream pattern-matching work: identifying the agreement and extracting cited changes from unstructured text.
A deterministic verification layer then checks every citation against the original document, validates the output schema, confirms semantic consistency between the clause and rule type, and verifies the source hash.
Only a verified change can reach the downstream policy engine.
The policy engine applies the reviewed rules across the live DataHub lineage and produces differentiated proposals. Humans remain responsible for approving consequential action.
How DataHub powers Covenant
DataHub is not a metadata source added to the side of the product. It determines what Covenant knows, what it evaluates, and where the resulting reasoning becomes durable.
| Point | DataHub’s role | Product consequence |
|---|---|---|
| Agreement registry | Stores governed agreements as native entities with structured properties | Covenant has a durable record to match the incoming amendment against |
| Agreement lookup | Executes the live governed-agreement query | Determines whether Covenant recognizes the document or must refuse it |
| Impact derivation | Resolves the governed source and reconstructs downstream lineage | The graph, not the client, decides which assets are affected |
| Usage context | Provides the properties used to classify each downstream use | Deterministic policy can produce different responses for different assets |
| Native writeback | Stores Covenant’s proposal, evidence, and status on each entity | The reasoning becomes durable, inspectable, and available to future DataHub queries |
Replacing DataHub with static JSON would remove the live registry, graph-derived affected set, usage context, native writeback, and read-back verification that the demonstration depends on.
Verified extraction
Covenant uses Amazon Bedrock with Claude Sonnet through structured tool use.
During agreement matching, the model receives exactly one available tool: lookup_governed_agreement.
Its output then passes through an independent verifier that:
- confirms every citation appears verbatim in the source;
- validates the response schema;
- checks that the proposed rule matches its cited language;
- verifies the source document hash;
- rejects failures with a named reason.
There is no bypass from model output to downstream propagation.
Verified writeback
Writeback proceeds sequentially for each entity through real backend states:
WRITING → WRITTEN → VERIFYING_MCP → MCP_VERIFIED → VERIFYING_SDK → SDK_VERIFIED → VERIFIED
These states come from the backend. They are not frontend timers or staged progress animations.
If one entity fails, previously verified entities remain complete and Covenant identifies the exact failed phase. Replaying the operation preserves stable entity identities and timestamps.
Challenges we ran into
Ensuring DataHub, not the browser, decides what is affected
Covenant’s thesis depends on the affected set coming from live DataHub lineage.
We derive downstream terminals only from DataHub-returned URNs and refuse client-supplied affected lists.
Adversarial tests confirmed that renamed URNs, altered fixtures, and DataHub outages fail closed without silently falling back to cached lineage.
Working around missing MCP fields without bypassing MCP
DataHub MCP Server 0.6.0 does not currently expose custom properties for Dashboard and MLModel entities.
We did not route the workflow around MCP.
MCP still resolves the source, derives every affected URN, and reconstructs every lineage path. A type-aware SDK reader retrieves only the missing terminal properties required for deterministic classification.
The boundary is explicit:
- MCP performs the graph work;
- the SDK retrieves the missing aspect details.
We filed the projection gaps as upstream DataHub issues.
Making fresh failures erase stale success
During integration, we found that a successful impact result could remain visible after a later attempt failed.
That is dangerous in a governance system because an old affected set can appear current.
Covenant now clears the prior impact projection before every live DataHub request. If the fresh request fails, the interface shows no affected set and reports the outage honestly.
Accomplishments that we are proud of
- One amendment produced five differentiated outcomes: continue, remediate, proposed stop, human review, and verified no impact.
- DataHub derived the affected set: Covenant never relied on a client-provided list of assets.
- Every extracted rule was independently verified: invented or malformed citations could not reach propagation.
- Every proposal was written back and read back: Covenant confirmed the final state on the native DataHub entity.
- The unresolved case remained unresolved: Covenant did not convert missing certainty into permission.
- The unrelated control remained untouched: the system proved that it could avoid unnecessary action.
- Extraction was stable across 10 consecutive runs: candidate identities and rule semantics remained consistent.
- The build generated upstream value: we filed two DataHub issues based on real integration gaps.
What we learned
Rationale is more durable than classification
A label alone does not survive staff turnover.
The durable governance asset is the combination of the triggering clause, affected use, accountable owner, lineage path, applied policy, and verification evidence.
Language models belong upstream of deterministic enforcement
LLMs are effective at extracting cited claims from unstructured agreements.
They are risky wherever an incorrect answer becomes durable or triggers consequential action.
Covenant therefore uses the model for interpretation, then places deterministic verification and policy enforcement between that interpretation and persistent governance state.
Failure behavior separates real agents from demo agents
A governance system must refuse invented citations, fail cleanly during outages, clear stale state, preserve uncertainty, and stop where human authority begins.
Producing an answer is not always success. Sometimes the correct output is to refuse one.
What is next for Covenant
Governed agreements as first-class DataHub entities
Agreements should have their own ownership, lifecycle, version history, structured obligations, and downstream relationships inside DataHub.
That would make contractual requirements queryable, governable, and inheritable across the metadata graph.
A broader qualification matrix
We plan to evaluate Covenant against more licenses, data-use agreements, regulatory notices, and contract formats while expanding its deterministic rule vocabulary.
Human-resolution feedback
When a governance lead resolves a HUMAN_REVIEW case, Covenant should preserve that decision and use it to update the classification vocabulary and policy mapping - without silently retraining the system.
Additional ingestion channels
Agreement changes also arrive through email, contract-management systems, regulator notices, and Slack. Covenant’s architecture begins downstream of ingestion, making those channels natural extensions.
Upstream contributions
We intend to contribute fixes and documentation for the DataHub MCP gaps discovered during the build.
Built With
- amazon-bedrock
- anthropic-claude-sonnet
- boto3
- datahub-core
- datahub-mcp-server
- datahub-python-sdk
- fastapi
- python
- react
- typescript