Sidq

• 

  Four agents, one catalog, no coordinator — DataHub their only shared state. Measured live: 18 examinations, 14 distinct, one killed mid-run.

• 

  DataHub's own shipped sample: 285 internal contradictions, concentrated in 5 assets, found by examining all 67 with read-only metadata.

• 

  Real engine output: a PII column removal blocked, with column lineage followed hop by hop to the live Looker dashboard it breaks.

• 

  The audit resumes through the catalog: receipts one run writes are what the next run reads, so coverage converges under a fixed budget.

• 

  Five gates collect evidence, one policy engine decides, the receipt goes back into DataHub. No model participates in a blocking decision.

Sidq refuses, repairs, and remembers

An analytics agent is about to change a PII-bearing model that feeds a critical dashboard. Sidq reads DataHub, traces the impact, and returns BLOCK before the agent acts. Its repair agent then proposes only changes whose resulting graph the deterministic engine can re-prove: in regression, a one-hop tag repair was refused because it moved the leak downstream; the proposal that passed covers the field-lineage closure — every column the marker reaches downstream that does not already declare it. On request, Sidq writes the result back into DataHub so the next agent inherits evidence instead of repeating an unsupported assumption. Run make gate-demo to re-derive the committed verdict and its hashes.

The problem

Agents increasingly plan from catalog metadata, but a graph stores claims, not proof that those claims agree. A stale schema, broken field lineage, missing owner, or unpropagated PII tag can turn useful context into confident automation built on a contradiction.

The problem is measurable in the platform's own example. A read-only examination of DataHub's shipped showcase-ecommerce sample checked all 67 datasets and found 285 internal field-lineage contradictions concentrated in 5 assets. One Power BI asset stores two fields but has 58 column-lineage edges; 57 target fields are absent from that schema. The truth report and machine-readable result make every count inspectable.

And the same read says who wrote them. Attributing each side of every finding to the ingestion run that persisted it — systemMetadata on the aspect read, one query parameter and no extra calls — shows the run that wrote the lineage naming the columns is the run that wrote the schema omitting them, in all five affected assets and every stored version: ten out of ten. So this is not two systems disagreeing, which is the usual reading of a catalog contradiction and would call for reconciling them; a single writer produced both halves. A second ingestion run reproduced the same shape exactly rather than repairing it, so it is deterministic output of an ingestion path rather than a transient failure — which is why the finding names a job to fix rather than a state to refresh. It is also why the total decomposes as five sibling assets times fifty-seven: one path, one defect, replicated across everything it emits.

And against reality, not only against itself

Comparing a catalog with itself finds contradictions. It cannot find the case where the catalog is internally consistent and simply wrong. Sidq also compares DataHub's claims with the system the data actually lives in.

make demo-break renames raw.customers.email to email_address in a live PostgreSQL source and re-ingests nothing. The catalog still says email. Sidq sees the graph and the source disagree and returns BLOCK on the context — before an agent plans against a column that no longer exists. sidq claims goes further: it turns each documented sentence about a field into a testable claim, compiles it to bounded read-only SQL, runs it against the source, and lets the deterministic engine judge the row count that comes back.

How you actually use it

Three surfaces, all from one engine:

An MCP server of our own, sidq-mcp, with exactly three tools — check_change (show it a diff, get PASS/WARN/BLOCK with evidence before you act), verify_context (is this asset lying to me?), and search_verified (search that only returns what was recently checked). Any MCP client attaches it in one line. A CLI — check, audit, claims, repair, verify, swarm, swarm-overlap, swarm-ledger, explain. A GitHub PR bot that comments the verdict, its provenance, the impact paths, and the exact command to reproduce it.

The audit is budgeted and resumable through the catalog itself. sidq audit --resume reads the receipts earlier runs wrote and skips every asset whose receipt still holds under the current policy hash, so the budget flows to assets no run has reached. Coverage converges run over run under a budget that never changed, and any Sidq instance resumes where any other stopped, because the state is DataHub — no state file, no sidecar, no daemon.

With writeback explicitly enabled, Sidq records queryable receipts through the official MCP mutation tools, and mirrors each verdict as a native DataHub Assertion in the Quality tab via DataHub's documented GraphQL custom-assertion API, with sidq as the assertion platform. The architecture and evidence boundaries are explicit; an unperformed comparison is never reported as clean.

The team case

Four auditor processes work one catalog concurrently with no coordinator, message bus, lock service, or shared filesystem. They divide work purely through the receipts they read and write in DataHub. The demo kills one worker mid-run; unfinished assets are not stranded because nothing was assigned. A fifth process that audited nothing reads the coverage ledger back from DataHub.

We measured it rather than asserting it, and the first measurement failed: 18 examinations covering 6 distinct assets, every asset examined by every surviving worker. Two defects, both found only because the number existed. Consequence scores are widely spread, so the per-worker ordering was shuffling inside bands that held one asset each. And a bounded read-after-write poll was cancelling in-flight MCP calls, which kills the server outright — a synchronous handler cannot observe cancellation, so mcp 1.29.0 responds twice and trips an assertion that unwinds the process. One timeout, and that worker could never write again. After both fixes the same run covers 14 distinct assets with 4 duplicated, write failures fall from 5 in 18 to 1, and DataHub holds 9 receipts across three workers instead of 4 from a single survivor. The coordination is now visible in the transcript: each worker reports the assets it skipped because a peer's receipt already covered them. And the duplication is spent rather than regretted: every asset two workers both reached is cross-checked — their examination digests are compared, and a live run reports how many independently agreed. Expect a different split each run; nothing is assigned, so which worker reaches which asset first is a race.

make swarm-demo. This is the "alone or as a team" case: DataHub is not merely an input or an output, it is the swarm's shared state.

Evidence

There is no LLM anywhere in the judged path. Only deterministic findings can block. The optional trained documentation reader may propose a bounded read-only SQL check and extend WARN coverage; it cannot grant permission or produce BLOCK. Same evidence, policy and commit produce a byte-identical verdict identified by policy_hash and commit_sha — you do not trust the tool, you re-derive its answer and compare hashes. Unavailable graph evidence fails closed.

The suite collects 1,180 tests: 1,179 passed, 1 optional integration skipped, above 84% enforced branch coverage, and guards fail the build when any published number drifts. The judge runbook is five commands, from an offline hash-locked replay to the live swarm. Property tests generate catalogs across eight platforms and five scripts and assert both halves: every planted contradiction is found, and no unplanted one is invented. Another 24 adversarial fixtures cover cycles, self-loops, 200-hop chains, 500-way fan-out, multilingual and emoji names, and malformed receipts.

Two pull requests are under public review: datahub#19017 and datahub-skills#81. Both open, neither merged — review evidence, not endorsement. Building on the official agent surface also surfaced defects we reported upstream with reproductions: a cancelled MCP request terminating the stdio server process (python-sdk#2416), and field lineage costing one round trip per column where one stored aspect read returns every edge (mcp-server-datahub#206).

Honest limits

Sidq proves only comparisons it performed. The sample ships no model SQL, so all 32 lineage_rot attempts were unverifiable; Sidq did not call them clean or rotten. Swarm work is at-least-once, not exactly-once: with no claim or compare-and-set primitive, two workers can safely duplicate a deterministic check. Receipts are latest shared state, not an append-only audit ledger. Writeback is off by default.

Try it

The live project is sidq.mlki.app — its buttons run the real commands on the host, and the times beside them are measured rather than estimated. Inspect the graph at datahub.mlki.app with sidq-judge@local.invalid / sidq-judge-2026. That credential is published so judges can inspect the same graph behind the submission, and it is deliberately powerless: Reader role, and addTag, createAccessToken, createPolicy, batchUpdateSoftDeleted and createInviteToken were each re-probed before submission and each returned 403 UNAUTHORIZED.

For local review, clone the Apache-2.0 repository and start with make gate-demo.

Disclosures

Sidq was created inside the submission period. AI coding assistants were used. Third-party material is data, not code: permissively licensed corpora with full provenance recorded in the repository. Apache-2.0.

Built With

• datahub
• dbt
• docker-compose
• github-actions
• looker
• mcp
• model-context-protocol
• numpy
• postgresql
• python
• pyyaml
• rest-api
• snowflake
• sqlglot