Kestrel.

Inspiration

Everyone agrees PII shouldn't land on a BI dashboard. That rule lives in a Confluence page nobody enforces, not because people are careless, but because the thing it describes isn't a property of any single table. It's a property of a path.

DataHub's Metadata Tests are good at what they do: one entity, one predicate, pass or fail. But the question governance teams actually ask is "does any column tagged PII reach a dashboard, through any multi-hop path, without being masked?" — and that is structurally inexpressible as a per-entity test.

In our own sample graph, patients.ssn is tagged PII at the source. Four transforms later the same value sits in a dashboard, untagged — the tag stopped, the data didn't. Every per-entity check on that path reports green.

What it does

Kestrel is Semgrep for your data catalog. You write governance rules as YAML; it enforces them across your lineage graph and writes findings back into DataHub.

• id: pii-reaches-bi subject: { entity_type: column, has_any_tag: [PII, Sensitive] } condition: lineage_reaches_type: [Dashboard, Chart] max_hops: 6 without_step: "mask|hash|sha2|redact|anonymi[sz]"

That without_step clause is the part no per-entity test can express: the path is fine if something masked the data on the way. Kestrel walks each route, suppresses the ones that pass through masking, and says so.

Four rules ship, three walk paths (including one that walks upstream, catching certified assets that depend on stale sources), one is deliberately per-entity so the contrast is honest rather than rhetorical.

How we built it

Python + Typer + Rich. Everything reaches DataHub through the MCP Server on an OSS instance.

Reads: search selects subjects, get_lineage walks table- and column-level lineage, get_entities resolves owners, get_dataset_queries pulls the real SQL that becomes evidence.

Writes, three layers per violation, because a scanner that only prints red text changes nothing:

Layer 01 matters most: tagging the exposure point means the analyst who opens that dashboard tomorrow inherits the finding without knowing Kestrel exists.

Two engines, one interface. The template engine is deterministic and LLM-free on purpose — a nondeterministic governance tool isn't a governance tool. The agentic engine is additive: it compiles plain English into the DSL and hands it to the deterministic engine, or investigates when the DSL can't express the rule — validating every URN it reports against the catalog so a hallucinated entity is rejected, not published.

update_description is Cloud-only, so nothing depends on it. This runs on the free thing.

Challenges we ran into

Three bugs surfaced only on first contact with a real DataHub, all invisible against fixtures.

The MCP SDK had renamed Tool.inputSchema to input_schema. We read it with getattr(tool, "inputSchema", None) — which didn't raise, it returned {}. Every tool appeared to advertise zero parameters, all four policies died at bind time, and the error pointed at the wrong layer entirely.

The honest one: we did not get a violation written back live. kestrel doctor confirmed the MCP connection, tool enumeration, and all three write tools exposed with mutations on — and all four policies executed against the live catalog without error. But showcase-ecommerce has no PII-tagged columns, so pii-reaches-bi correctly found nothing. The integration and tool surface are verified; the final write on a live violation isn't. That's a data gap, not an architecture one.

Accomplishments that we're proud of

The agent produces a reusable artifact, not an answer. --ask "no dashboard should depend on a deprecated table" compiled that sentence into a policy file, saved it, and found revenue_ops → billing_rollup → legacy_billing two hops up with full evidence. That file goes into code review and runs every night forever. That's the line between an agent and a chatbot.

And pip install -e . && kestrel scan finds six real violations in under a second — no DataHub, no key, no Docker.

What we learned

Silent defaults are worse than crashes. A default on a lookup you expect to succeed converts a loud failure into a quiet lie.

The failure mode that scared us most was a green checkmark. When every policy errored, the report printed No violations. Every policy passed. A governance tool reporting a clean bill of health over a catalog it never read is worse than one that crashes, because someone will trust it. Errored policies now surface as incomplete — explicitly not a pass.

Confidence you can't audit is decoration. Column-level lineage often isn't populated. Kestrel falls back to table-level and says so on the specific hop.

What's next for Kestrel.

Live write-back proven end-to-end against a catalog with column-level PII tags. A GitHub Action so policies gate merges the way Semgrep does. The policy pack contributed upstream as a DataHub Skill.

Built With

• anthropic
• claude
• data-governance
• datahub
• docker
• github-codespaces
• gsap
• mcp
• model-context-protocol
• nextjs
• pydantic
• pytest
• python
• react
• rich
• tailwindcss
• typer
• typescript
• yaml