Half-Life: Memory Integrity for DataHub Agents

• 

• 

• 

Inspiration

DataHub published a post called AI Agent Memory: Why Quality Is a Data Problem that makes a sharp argument:

An agent with access to incorrect memorized facts becomes more risky than a stateless agent, because errors lock in rather than expire.

It lists four primitives for trustworthy agent memory, lineage, business glossary, change detection, ownership, all of which DataHub already ships. Then it names three problems it leaves open:

1. Keeping agent memory invalidated when upstream definitions change
2. Preventing silent data drift in agent memories
3. Routing escalations correctly when organizational ownership changes

Before building anything, we went looking for what was genuinely missing rather than what was easiest to demo. Agents writing memories into DataHub is solved and demonstrated, a LangChain agent saving a query as a context document was shown at DataHub's own town hall.

Nothing takes a memory away.

So catalogs accumulate confident, well-formatted, permanently wrong institutional knowledge. The column was renamed in March. Finance redefined "Order Total" to exclude discounts. The engineer who owned the table left in June. The memory still reads exactly as authoritative as the day it was written, and the next agent inherits it.

What it does

Half-Life gives agent memories a lifespan.

Memories are stored as DataHub Documents carrying typed provenance in structured properties: the URNs they were derived from, plus a fingerprint of that metadata's state at the moment of writing. Half-Life then re-checks them using two independent signals:

• DataHub's Timeline API, which already grades every change for backwards compatibility. A MAJOR change to a dependency expires the memory outright.
• Fingerprint drift, a dependency whose state moved with no timeline event to explain it. This is the silent-drift case: lineage rewiring and unsupported entity types leave no timeline trail.

Verdicts are written back into the graph:

Outcome & What happens VALID : Score and validation watermark updated SUSPECT : Tagged, score reduced, returned by recall with a warning showing the before/after EXPIRED : Unpublished, recall can no longer return it tagged, and an audit document written

Expiry is structural, not advisory. An expired memory is removed from circulation, so an agent physically cannot inherit a fact that stopped being true. It has to re-derive from the catalog, which is the correct behaviour.

Ownership changes never invalidate a memory (the facts are unaffected) but they re-route escalation to whoever owns the dependency now, not whoever recorded it.

Every state change writes an audit document into DataHub, linked to both the memory and the asset that changed, so the causal chain lives in the catalog rather than a log file.

Precision is the whole point

A tool that turns everything red after any change is as useless as one that turns nothing red. In the demo, three upstream changes land and of five memories: one expires, one becomes suspect, one has its escalation re-routed, and two stay untouched.

The green rows are the claim.

The loop, end to end

Asked on camera: "can I get intraday stock levels from the inventories table?":

1. recall withheld the memory that answered it: "removal of field quantity_on_hand, re-derive this from the catalog; do not trust it."
2. The agent went back to DataHub's MCP server, found the column deleted and a Daily freshness SLA
3. It recorded a corrected memory in its place

The wrong answer left circulation and a right one replaced it, with no human intervention.

How we built it

Python 3.10+, against DataHub Core v1.5.0.6 with the showcase-ecommerce datapack (~1,050 entities across Snowflake, Looker, PowerBI, Tableau, dbt, Spark, Postgres, S3).

• Timeline API for compatibility-graded change detection, reusing DataHub's own grading rather than diffing metadata ourselves
• Documents as memory storage, structured properties for provenance
• GraphQL for entity snapshots, DataHub Python SDK for write-back
• An MCP server (recall / record_memory / validate_memories / explain_memory / memory_ledger) designed to run alongside DataHub's own: DataHub answers "what is true right now", Half-Life answers "what have we learned before, and does it still hold"
• FastAPI ledger, Typer CLI, 33 unit tests over the pure severity logic

record_memory rejects an empty dependency list. A memory with no recorded provenance can never be invalidated, which leaves it permanently authoritative, worse than not storing it at all.

Challenges we ran into

Four DataHub API behaviours contradicted the documentation. Each was found by running against a live instance, and each is now handled in code and pinned by a regression test:

1. The Timeline API ignores its start parameter and replays an entity's entire history. This silently destroys precision, every freshly recorded memory gets judged against the ingestion that created its own dependencies, and goes suspect immediately. No error, no warning.
2. The OWNERSHIP category returns HTTP 500 on every URN, including ones that don't exist. The "legacy" OWNER alias is the one that works, the reverse of what the docs say.
3. Response shape differs: a bare transaction array, not the documented changeTransactions wrapper, with entityUrn/operation/modifier instead of target/changeType/elementId.
4. Editing a glossary term's definition is reported as DOCUMENTATION, not GLOSSARY_TERM, that category is reserved for terms being attached to entities. Redefining a term changes what every memory built on it means, so we treat documentation changes on glossary terms as semantic while the same edit on a dataset is not.

All are reported upstream: datahub-project/datahub#19053.

Separately, GMS ingests asynchronously, which made the documented command sequence racy, seed returned before its own memories were readable, so a following validate silently operated on a partial set. Printing "give it a few seconds" wasn't good enough: anyone following the README verbatim would see the wrong result. Each step now polls until its own effect is observable.

What we learned

The research mattered more than the code. Our first idea, a cross-platform breaking-change PR guard, was abandoned after we found DataHub already ships acryldata/dbt-impact-action, Impact Analysis, Schema Assertions, Subscriptions and Data Contracts. Text-to-SQL, documentation improvement, glossary propagation and quality agents are all shipped too.

Finding the genuinely unsolved problem took longer than building the solution, and it was the right trade. docs/research.md records what was ruled out and why.

The second lesson: verify against a running instance, never the docs. Three of our four blocking assumptions held; the fourth was false in a way that produced no error and would have quietly made the tool useless.

What's next

• Scheduled validation as a DataHub Action rather than a CLI invocation
• Lineage-aware invalidation, expire memories when an upstream dependency of a dependency changes, not just direct ones
• Upstreaming the datahub-memory-integrity skill and the Timeline API fixes

Built With

• datahub
• docker
• fastapi
• graphql
• kafka
• mcp
• model-context-protocol
• opensearch
• pytest
• python
• rest-api
• typer
• uv