RIFT — git diff for business logic

$410,258.29 explained in full, and PENDING_HUMAN_REVIEW — written to the asset in DataHub. RIFT proposes; a human decides.
Measure each cause alone and you get $819,626 — 2× the real gap. Shapley values fix the overlap, so the residual is $0.00.
One command against a live DataHub: 9 reads, 7 writes, 10 query executions, and every dollar of the gap accounted for.
RIFT — git diff for business logic. Two dashboards disagree about revenue; RIFT proves exactly why, in dollars.
Each cause, its dollar contribution, and whether it's a probable defect or a deliberate modelling choice machine-readable on the asset.
RIFT tags both assets and files a pending proposal. It never certifies either one that decision belongs to an owner.
The full reconciliation published into DataHub's knowledge base, alongside the two datasets it compared.

Inspiration

We asked an AI agent what our revenue was.

It searched the catalog, found the table tagged Tier-1 and owned by Finance, noted that it was "explicitly designated for board reporting," and answered. Confidence: high.

It was off by $410,258.29, and it had no way of knowing. A second team publishes the same metric from different SQL, and nothing in the catalog says so.

That is the part that bothered us. Not that two definitions of revenue exist — every company has that. The problem is that a catalog full of perfectly accurate metadata can still hand an agent a confidently wrong number, because "these two disagree, and here is by how much" is not a thing catalogs record.

The usual fix is a person. Someone spends two days reading both queries, finds the cause, posts it in Slack, and it evaporates. Six weeks later the numbers drift again and somebody else does the same two days.

So we wanted two things. An answer that comes with arithmetic instead of an opinion — and an answer that stays in the catalog, so the next agent doesn't have to be wrong first.

What it does

RIFT is an agent that audits a data catalog for metrics that disagree, proves the cause in dollars, and files a proposal for a human to decide.

Finds the collision. Reads DataHub over the official MCP server and scores asset pairs on shared distinctive columns, glossary terms and upstreams. Our pair scored 0.867.
Diffs the logic structurally. Parses both queries into syntax trees and classifies every real difference — join type, aggregate expression, filter predicate, date bound. A textual diff is useless here; formatting changes constantly without changing meaning.
Measures each difference by executing it. This is the part that matters. For each difference, RIFT rewrites the query with that single change applied and re-runs it. Ten query executions. Nothing is estimated, and no language model is asked to guess.
Handles the overlap. Causes interact — a permissive join and a defaulted value in an aggregate hit the same rows. Measure each alone and add them up and you get $819,626.62, which is 2.00× the real gap. RIFT attributes with Shapley values over all orderings, so contributions are order-independent and sum to the true difference. The residual is $0.00.
Separates defects from choices. Excluding trial accounts is a legitimate business decision. Replacing a missing exchange rate with 1.0 inside an aggregate is a bug — it silently books foreign revenue at 1:1. RIFT derives that distinction from query structure and labels it a hypothesis, not a verdict.
Writes the answer back and proposes. Publishes the reconciliation as a document, attaches machine-readable findings to both assets, tags them, and files a pending proposal. It never certifies an asset. There is a test that fails if any write ever claims canonicity on RIFT's own authority.

On our catalog it explained 100% of a $410,258.29 gap: $175,635.14 from a missing FX rate defaulted to 1.0, $175,635.14 from a LEFT join that should have been INNER, and $58,988.01 from counting trial accounts. It found the 355 rows where an Australian subscription billed at 4677.15 AUD was booked as $4,677.15 USD.

The result we did not expect

We asked an AI agent — connected to our live DataHub over MCP — the question a finance lead would ask: "What was our MRR? I need a single number for the board deck, and I need to know whether I can trust it."

Before RIFT had ever run, the agent answered:

"Use the latest month's SUM(mrr_usd) from analytics.finance.revenue_finance. It is the Finance-owned, Tier-1 model explicitly designated for board reporting. Confidence: High that this is the correct source."

It picked one of two disagreeing models, called it authoritative, and never mentioned the other one existed.

Then RIFT ran once and published its findings. We asked the same agent the same question, with the same catalog tools — and it still had no access to RIFT:

"Do not use $410,258.29 — that is the documented gap between Finance and Growth models... The reconciliation fully explains the gap, with $351,270.28 attributed to probable defects in the Growth model and a human decision still pending."

The only variable was whether RIFT had written to the catalog. So RIFT does not just answer the question once — it makes the catalog itself safer for every agent that reads it next. Both transcripts are in examples/.

How we built it

DataHub MCP server for every read and write — search, get_entities, list_schema_fields, get_lineage, get_dataset_queries, save_document, add_structured_properties, add_tags.
RIFT is also an MCP server, exposing find_metric_collisions, reconcile_metrics, compare_sql and publish_reconciliation, so any agent can ask it which number to trust. In live mode it reads the real catalog, not a fixture.
A DataHub Skill (skills/rift-metric-reconciliation/) in the official registry format, teaching any agent the workflow — including the practical traps we hit.
sqlglot for structural diffing, DuckDB for execution, and a Shapley implementation over all orderings for attribution.
Runs against a live DataHub v1.7.0 (OSS, Docker) and against a checked-in fixture that reproduces the whole demo in about two minutes with no cluster.

Challenges we ran into

Our DataHub integration had never actually worked. We wrote it against the shapes you would guess from DataHub's REST vocabulary, and the first live run failed on every single call:

search takes num_results, not count, and has no entity-type filter — it returns glossary terms alongside datasets, so the caller must filter by URN.
get_lineage takes upstream: bool, not direction: "UPSTREAM".
Every write had different argument names (tag_urns/entity_urns, property_values as lists keyed by property URN, document_type from a closed enum).
Our ingest script nested schema field types one level too shallow, so it could not have worked against any DataHub.

And our tests passed the whole time, because they asserted which tools we called and never with what arguments. That is the lesson we will keep: a stub that only checks tool names verifies almost nothing. The tests now pin argument shapes against payloads captured from a live server.

DataHub stores a NUMBER structured property at single precision. We wrote 410258.29; it stored 410258.28125. For a tool whose entire output is a dollar figure, the asset page then disagreed with the document attached to the same asset. Money is now carried as exact decimal text.

Other sharp edges: tags must already exist before add_tags will accept them (ingestion auto-creates them; the GraphQL path does not), save_document with a URN that does not exist returns success and writes nothing, and a structured property's Elasticsearch field mapping survives deleting the property — so changing its valueType needs a volume wipe.

The open-source MCP server has no lifecycle-proposal tool. Eighteen tools with mutations enabled, and none of them proposes anything. "The agent proposes, a human decides" is the whole safety story for agentic writes, and OSS DataHub has no primitive for it. We express the proposal as a rift:proposal-pending tag plus rift.decision = PENDING_HUMAN_REVIEW — honest and visible, but not a workflow. If we could fix one thing in DataHub, it would be this.

What we learned

That the write-back mattered more than the tool. We built RIFT to answer a question, and discovered that publishing the answer into DataHub changed how a completely separate agent behaved. Catalog metadata is not documentation — it is the context every agent inherits.

What's next

The engine is warehouse-agnostic but currently executes against DuckDB. Snowflake and BigQuery adapters are the obvious next step, and they are the only thing standing between this and running against a real company's catalog.