Tripwire — Adaptive Data Change Safety

The problem

A SQL change can compile, pass unit tests, and still silently change a feature value that changes a model prediction—and the production agent acting on that prediction. Existing impact tools show what might be affected. They do not execute downstream behavior, produce a minimal proof, verify a repair, or teach the organization not to repeat the same failure.

What Tripwire does

Tripwire is an adaptive change-safety agent built on DataHub. For an exact dbt/SQL revision, it:

1. Resolves the changed model through the compiled dbt manifest and records normalized SQL AST facts.
2. Reads the live DataHub graph through the official MCP Server: entity metadata, real schema, ownership, tags, lineage paths, ML model/deployment context, and prior protection memory.
3. Selects critical downstream checks from that context.
4. Executes the baseline and candidate through DuckDB, a hash-pinned logistic fraud model, and a deterministic Fraud Review Agent.
5. Returns one honest verdict: UNSAFE, SAFE_WITHIN_SCOPE, or UNVERIFIED.
6. Minimizes a behavioral regression into a portable counterexample witness.
7. Generates a remediation and labels it verified only after replaying the same critical checks.
8. After explicit human approval, writes a versioned Change Passport and reusable Protection back to stable DataHub URNs and attaches the protection to every affected asset.
9. On a different future SQL change, retrieves that protection through live DataHub context and catches the regression automatically.

The three-part demonstration

• Unsafe change: a null-handling edit changes fraud features, model probabilities, and agent actions. Tripwire blocks it with exit code 1 and a minimal TX-009 witness.
• Learned catch: the approved witness is persisted in DataHub. A different SQL expression recreating the defect is blocked with LEARNED_PROTECTION_VIOLATED.
• Safe control: an additive reviewer-note change runs through the same graph, model, agent, and protection. All critical checks pass and Tripwire returns SAFE_WITHIN_SCOPE with exit code 0.

This is a causal control, not a red-only demo.

Why DataHub is foundational

Tripwire does not use DataHub as a decorative catalog lookup. DataHub supplies the identity and context required to decide what must be tested:

• exact dataset and field schema;
• ownership and review routing;
• downstream dataset → ML feature → model → deployment → agent lineage;
• criticality metadata;
• prior human-approved protections.

Tripwire also contributes results back so the next developer or agent inherits the knowledge. Without the DataHub graph, the agent cannot resolve the real asset, select critical consumers, route ownership, or retrieve organization memory.

The public evidence bundle was captured against DataHub OSS 1.7.0 using mcp-server-datahub 0.6.0. It contains source-hashed MCP facts, complete coverage accounting, seven lineage paths, ten seeded entities, a model artifact hash, DataHub writeback receipts, and machine-readable Passports.

Technical execution

• Python 3.11–3.13, Typer, Pydantic, HTTPX
• DataHub OSS/Core 1.7.0 and DataHub MCP Server 0.6.0
• dbt-duckdb, DuckDB, SQLGlot
• executable logistic-regression model artifact and deterministic review agent
• GitHub Checks and GitHub Actions
• public evidence console built with React/vinext and deployed on Cloudflare Workers through Sites

The repository enforces Ruff, strict Mypy, 50 tests, 90.31% branch coverage, package construction, console production build, rendered-output tests, ESLint, and a dedicated Tripwire / Change Safety Check on every pull request. Multiple changed models fail closed; unrelated PRs report explicit not-applicable status and make no safety claim.

Judge-ready evidence

The repository includes:

• a one-command live DataHub quickstart;
• unsafe and safe public control PRs;
• committed live MCP/DataHub v2 proof;
• sample Change Passports, compact Check reports, witness replay, remediation, active Protection, and writeback receipt;
• a reusable Agent Skills-compatible Tripwire workflow;
• an Apache 2.0 license and complete setup instructions.

The hosted console exposes the current evidence chain without requiring judges to run the full stack.

Open-source contribution

We generalized the core workflow into a vendor-neutral DataHub Change Safety Skill and opened it upstream:

https://github.com/datahub-project/datahub-skills/pull/122

The contribution adds exact-change binding, DataHub context coverage, executable downstream validation, three-state verdicts, a Change Passport contract, human-gated protection memory, routing, command integration, and documentation. It contains no Tripwire branding.

Honest scope

Tripwire deliberately supports one fraud vertical slice exceptionally well rather than pretending to be a universal SQL safety engine. SAFE_WITHIN_SCOPE is bounded by the DataHub context and critical consumers actually executed. Unsupported or ambiguous changes return UNVERIFIED rather than a false pass.

What's next

Next steps are additional dbt models, warehouse execution adapters, feature-store/model-serving adapters, batch assessment for independently changed models, and native organization policies for how Change Passports and Protections are represented in DataHub.

Built With

• cloudflare-workers
• datahub-mcp-server
• datahub-oss
• datahub-skills
• dbt
• duckdb
• github-actions
• pydantic
• python
• react
• sqlglot
• typer
• vinext