Probate

513 edges re-checked: 266 confirmed, 73 dead, 29 undeclared, 145 not judged.
Per-edge verdicts, filterable by state, domain and owner.
A dead edge with its evidence and the date it stopped being true.

Inspiration

DataHub tells you table A feeds table B. It never tells you whether anyone has used that path since the day it was declared. So lineage rots quietly, and every migration, every deprecation, every impact analysis inherits the rot.

We wanted to know: can you re-check a catalog's own claims against what actually happened?

What it does

Probate re-checks every lineage edge. Per edge. With a date.

One run against our demo catalog — 618 tables, 4 platforms, 6 databases, 1,476 API calls:

Verdict	Edges	What you do about it
`confirmed`	266	declared and measured — leave it alone
`dead`	73	declared, zero traffic in the window — and it tells you since when
`missing`	29	real query traffic, never declared — lineage your catalog doesn't know it has
`unknown`	145	no usage data at all — no statement made
total	513

That last row is the whole product. Every lineage-cleanup tool we found would have reported those 145 edges as dead and invited you to delete them. Probate refuses: we could not look is not nothing happened. It was our own bug first.

The projects we found use usage data to weight edges they already believe. This tool uses it to test the edge itself.

We surveyed the field and found seven projects that read both lineage and usage. All seven attach usage to an edge as a weight, a hint, or an evidence string on lineage they already accept — none asks whether the edge should exist. That is a claim about seven projects, not about the world: GitHub's search does not index READMEs and private repos are invisible, so it is a lower bound, not a census.

It proves it re-measures, instead of asserting once

Four states existing is easy. An edge moving between them when the world changes is the actual claim — so we measured it: 7 of 7 state transitions proven live. Declare an edge → dead. Seed a query → confirmed. Cut the usage data → unknown, not dead. Every mutation went through the same journalled write path as the CLI and was taken back inside the same run: 3 journal lines added, 3 reverted.

The model has to cite, or it is overruled

There is exactly one model call in the whole project: judging which deviation matters — a dead edge into the annual-close table is not a dead edge into a nameless staging table. It runs under a citation duty: no verbatim quote from the metadata it was given, no judgement, and the deterministic finding stands unchanged. On our seeded lab the rule fired against the model on one of three offered edges. We publish the abstention rate rather than tuning the fixture until it looks better.

How we built it

Reads declared lineage (upstreamLineage), measured usage buckets (usageStats, getTimeseriesAspectValues) and query co-occurrence (QuerySubject) over GraphQL and REST/OpenAPI v3.
Reads the same thing over DataHub's own MCP server (get_dataset_queries, pinned mcp-server-datahub==0.6.0) as a second, switchable transport — and we measured what that costs: identical verdicts across all four states, 0.075 s end-to-end over GraphQL vs. 9.661 s over MCP, of which ~9.4 s is session start; 6 tools by default, 18 with TOOLS_IS_MUTATION_ENABLED=true. Where MCP stops is precise: queries.gql in 0.6.0 comments out created and lastModified.time, so query entities carry no timestamp — co-occurrence over MCP yes, the age judgement no.
Writes back so the next agent inherits it: testResults (the evaluator DataHub displays but doesn't have), a probate.dead_since marker dated from the measurement not the run, and merged lineage edges — dry-run by default, journalled, with a real reverse gear that deletes aspects that didn't exist before instead of writing back an invented None.
Watches continuously via acryl-datahub-actions on real Kafka (80 events received, 1 relapse detected and correctly refused as a dry run).

Python 3.11 pinned through uv · DataHub OSS quickstart · MCP server 0.6.0 · acryl-datahub-actions + Kafka · Claude behind an injectable str -> str transport so the judge is fully testable offline · plain HTML/CSS/JS console, no framework · pytest · Apache 2.0.

Challenges we ran into

harness/audit_claims.py re-derives every number in our prose from the committed run artifacts and exits 1 on any drift. 140 numbers are under that gate right now. It runs offline, without a DataHub instance, in seconds.

We ran that gate against ourselves and lost, repeatedly — and published the corrections instead of the originals:

Three claims we had made about DataHub's own MCP server (@latest is 0.4.0, zero mutation tools, ~54 s telemetry per call) turned out to be wrong five days later. We re-measured and replaced them.
Our own write path was destroying column-level lineage — the exact failure we accuse sql_parsing_aggregator.py of.
Our citation rule was satisfiable with a one-character quote.

Accomplishments — upstream contributions, filed not just prepared

Seven contributions to DataHub, all filed on 2026-08-10, every one re-measured against a live instance the same day:

usageStats.aggregations.uniqueUserCount is always 0 while buckets in the same response report 4 and 3 — a wrong number on the dataset page, not an error — datahub#19066
Dataset.usageStats returns HTTP 500 "Name is null" when the nullable range argument is omitted — datahub#19067
Root cause behind an existing report: AspectResource unboxes a null limit — comment on datahub#19016
Query entities carry no timestamp over MCP — a client defect, not a backend limit — mcp-server-datahub#209
Docs: usageStats.buckets is a dense calendar grid — 125 buckets for 8 measured days — datahub#19068
Docs: OpenAPI v3 get-aspect returns only the newest row for timeseries aspects — datahub#19069
A correction to someone else's issue — three MCP-server claims we had copied into our own notes as facts, re-measured and corrected — comment on mcp-server-datahub#167

Number 3 is deliberately a comment, not an issue: the bug was already reported, and a duplicate helps nobody. Number 7 is the one we would point a juror at.

What we learned — honest limitations, up front

All usage data in every run is seeded (harness/seed_lineage_lab.py) — DataHub's sample catalog ships none, live-checked: totalSqlQueries: null, buckets: []. Every artifact carries "seeded": true. No real warehouse has ever been connected. The load test does not demonstrate the two-stage cost saving it was designed to measure, and says so. The watcher's re-evaluation could not be shown changing a verdict, and says so.

A tool whose claim is "assert nothing without evidence" doesn't get an exception for itself.