Bulwark

The Problem

AI agents are getting write access to production data stacks. They propose schema changes, drop columns, alter types — and the SQL executes cleanly. No error is thrown. No alert fires. Three production ML models depended on that column, and nobody finds out until churn scores go quietly wrong a week later.

The failure is that nothing appears to break until it's expensive.

What Bulwark Does

Bulwark is a two-agent governance system built on DataHub's lineage graph.

A worker agent proposes real data stack changes — dropped columns, altered types, renamed tables. A governor agent intercepts each proposal, reads the full downstream lineage graph via mcp-server-datahub, and issues a verdict before anything reaches the database:

• CONTAIN — blocked. A production ML model or dashboard depends on this column. The change cannot ship until the dependency is resolved.
• WARN — flagged. No registered dependents, but the table is production and lineage coverage is never 100%.
• ALLOW — approved. No downstream dependents found.

The governor then writes its verdict back into DataHub as metadata tags (urn:li:tag:bulwark-contain, urn:li:tag:bulwark-warn) so every subsequent agent or engineer inherits the ruling.

Why a Second Agent, Not Better Prompting

The worker cannot check its own work. It shares the exact blind spots that produced the proposal. Bulwark splits authority: the worker has permission to propose, the governor has permission to block, and neither can override the other.

The governor's veto is derived dynamically from the DataHub graph — not from a policy file a human wrote down. Which means it catches dependencies nobody thought to document.

The Accidental Catch

During development, the governor flagged the worker's own code: the worker had proposed converting a money column (billing_amount) to a date type. The governor caught it as a type mismatch before it ran. We fixed the worker's heuristic as a result. This was not a planted scenario — it was the governor doing its job on input we didn't design for, which is the entire claim of the product demonstrated accidentally and therefore credibly.

How We Built It

• DataHub: real instance running in a GitHub Codespace (4-core/16GB), with a healthcare dataset containing a real ML model (billing_risk_model) added via DataHub's write APIs through a real training-job hop — the correct way DataHub models "trained on"
• Lineage reads: mcp-server-datahub for all governor reads, giving the governor access to the full context graph
• Lineage writes: acryl-datahub SDK for verdict write-back, tagging assets and writing Incident aspects so the graph accumulates decisions
• Worker: Python agent proposing schema changes against real DataHub schema field tags
• Governor: Python agent with Claude as the reasoning layer, reading multi-hop lineage before every verdict
• Frontend: Next.js 16.2 review queue where human operators resolve CONTAIN verdicts — accepting, applying generated migrations, or overriding with a recorded justification

What We Learned

The official DataHub sample packs (nyc-taxi, healthcare) contain no ML model — both are quality/freshness pipelines. Protecting production models is the core challenge prompt, but it is not testable out of the box. We added one via the real write APIs, documented it plainly in log.md and the README, and framed it as a finding: the sample data gap is itself useful feedback for the DataHub team.

We also learned that mcp-server-datahub version skew causes silent failures — the search index lag between write and query is real and needs a documented wait or retry. Both findings are in feedback.md.

Challenges

• DataHub Core requires 8GB RAM minimum. Local Docker on a 16GB machine left too little headroom. Solved by moving to a GitHub Codespace and tunneling — which also made the demo more reproducible.
• Column-level lineage in DataHub is sparse. The governor uses dataset-level lineage with production/ML classification rather than relying on column-to-column edges, which are rarely populated in practice.
• The Claude CLI times out intermittently on sequential calls. Fixed with a retry-then-fallback pattern in the governor, with the fallback clearly labeled so fixture output is never presented as real.

Built With

DataHub · mcp-server-datahub · acryl-datahub SDK · Claude (claude-cli) · Next.js 16.2 · TypeScript · Tailwind CSS · Python 3.11 · GitHub Codespaces

Built With

• apache2.0
• claude
• codespaces
• css
• datahub
• github
• mcp-server-datahub
• next.js
• python
• tailwind
• typescript