MetaMender

Inspiration

Metadata catalogs rarely fail in one dramatic event. They decay through hundreds of small gaps: an owner disappears, a description is blank, or a sensitive-looking field is never classified. Search quality drops, accountability becomes ambiguous, and governance teams spend their time chasing spreadsheets instead of improving the catalog.

We wanted a steward that could perform repetitive investigation and remediation without turning an AI model into an unchecked metadata administrator.

What it does

MetaMender scans a DataHub catalog through the official DataHub MCP server and produces a risk-ranked governance work queue. It detects missing owners, missing descriptions, PII-looking fields without the configured PII glossary term, and orphan cleanup candidates.

For each actionable finding, MetaMender explains the evidence and the exact DataHub change it proposes. A code-enforced terminal gate requires a fresh explicit "yes" for that single finding. Only then can it call add_owners, update_description, or add_terms. After mutation, it independently re-reads DataHub and writes Markdown and JSON audit records containing the finding, human decision, mutation arguments, DataHub acknowledgement, and verified before/after state.

With ANTHROPIC_API_KEY, Claude Agent SDK can drive the scan and fix tools. A provider-neutral OpenAI-compatible adapter now supports local Ollama models and hosted endpoints such as Kimi/Moonshot through the same guarded tools. With no model configured, a deterministic orchestrator drives the same workflow. In every mode, a model cannot invent a write target, batch approvals, or bypass terminal confirmation.

How we built it

MetaMender is a TypeScript agent built around the published mcp-server-datahub package, pinned to version 0.6.0. It keeps one long-lived MCP stdio session and uses search, get_entities, list_schema_fields, and get_lineage for discovery. Approved repairs use add_owners, update_description, and add_terms; verification performs fresh read calls rather than trusting a mutation acknowledgement.

The agent layer exposes scan_governance_gaps and apply_governance_fix as in-process tools. The apply tool is wrapped by a code-enforced confirmation gate shared with the deterministic fallback. Findings must come from the latest scan and are keyed by exact URN, kind, and optional column.

The repository includes deterministic detectors, bidirectional lineage-based orphan review candidates, configurable owner and glossary targets, mutation planning, independent verification, dual-format audit reporting, 88 passing default tests, and four real MCP integration checks against a local DataHub quickstart. We also ran a zero-API-cost Qwen 2.5 7B session through Ollama: the model called scan_governance_gaps against the live catalog, surfaced the missing warehouses description, and produced an audit with one finding and zero writes. The dependency audit reports zero known vulnerabilities. The committed adversarial evaluation passes 8/8 fixtures with 100% confirmation adherence, 0 unauthorized writes, 0 false-success reports, and 21 OpenTelemetry spans covering scan, confirmation, mutation, verification, and audit stages. Reproduce it with npm run eval; methodology and artifacts are linked from docs/agent-evaluation.md.

For the open-source bonus, we submitted a reusable read-only /datahub-audit skill to datahub-project/datahub-skills: https://github.com/datahub-project/datahub-skills/pull/70. It fills an existing routing gap and is independent of MetaMender's implementation.

Challenges we ran into

DataHub metadata has both ingested and editable overlays, so a description can look missing if only one aspect is checked. Column glossary terms have the same issue. Every detector and verifier resolves effective state instead of checking a single field.

Prompt instructions alone were not a sufficient safety boundary. We enforced target provenance and per-item confirmation in code, made EOF fail closed, prohibited batch approval, and required a fresh DataHub read before reporting success.

Finally, MCP tool startup is expensive when a new server is spawned for every call. MetaMender initializes one pinned server process and reuses the session for the entire round.

Accomplishments that we're proud of

• A real read → explain → approve → write → verify workflow against DataHub OSS.
• Code-enforced one-finding/one-confirmation safety shared by LLM and deterministic modes.
• Evidence-backed PII candidates with downstream-aware severity.
• Human-readable and machine-readable audit trails.
• 88 default tests plus four passing real DataHub MCP checks.
• A provider-neutral model path proven with local Qwen/Ollama and compatible with Kimi/Moonshot.
• 8/8 adversarial safety fixtures, 100% confirmation adherence, 0 unauthorized writes, 0 false-success reports, and 21 OpenTelemetry spans.
• A validated, reusable datahub-audit skill contribution submitted as datahub-project/datahub-skills#70.

What we learned

A useful governance agent needs more than mutation tools. It needs clear evidence, effective-metadata semantics, target provenance, human authority at the exact point of change, and independent verification. MCP made the DataHub integration portable; the hard work was designing the boundary around writes.

What's next for MetaMender

We will evolve the current environment-configured owner and glossary targets into versioned policy profiles, resolve ownership through domains and teams, add bounded scheduling and trend reports, and support additional governance dimensions such as assertions and structured properties. The submitted datahub-audit skill can make read-only catalog audits reusable without installing MetaMender.

Built With

• claude-agent-sdk
• data-governance
• datahub
• docker
• model-context-protocol
• typescript
• vitest
• zod