TRIPWIRE

Tripwire warns on GitHub PRs before you merge real DataHub lineage, downstream dashboards & owners, then writes pre-merge risk back into the catalog.

Comment

Tripwire

Your data catalog already knows this PR will break three dashboards. Nobody asked it.

Built for DataHub's Agent Hackathon. Primary category: Metadata Aware Code Generation and Development. Secondary: Agents That Do Real Work.

Public repo: https://github.com/ogawde/tripwire Demo PR: https://github.com/ogawde/tripwire/pull/2

Inspiration

Most data incidents do not start with a huge refactor. They start with a one line PR.

Someone renames a column, tweaks a dbt model, or “cleans up” a field that looks unused. CI is green. Reviewers approve because the diff is tiny. The merge goes through. A few days later a revenue dashboard is empty, a weekly exec report is wrong, or an analytics model is quietly reading stale logic. The on call engineer then spends hours reconstructing what depended on that table by searching Slack threads and Looker folders.

The frustrating part is that the answer already existed. Data catalogs like DataHub already track lineage, ownership, and downstream dashboards. The catalog knew what would break. It just never got a seat at the merge decision.

That gap is what Tripwire is about. Not another place to go look things up. A warning that shows up in GitHub, right where people decide to merge, using the lineage graph the company already maintains.

What it does

Tripwire is a GitHub bot for data pull requests.

When a PR touches files under demo-pipeline/, Tripwire:

  1. Maps the changed file to a real DataHub dataset (for example Snowflake orders).
  2. Looks up downstream lineage from a committed snapshot of DataHub's showcase-ecommerce graph.
  3. Posts a PR comment with the blast radius: dashboards, models, hop distance, and owners.
  4. In live mode, writes that assessment back into DataHub as a structured property called Tripwire Pre Merge Risk, including severity and a link to the PR.

The comment is meant to read like a careful teammate, not an alarm flood. Example shape:

  • this PR touches orders on Snowflake
  • it affects 3 downstream dashboards and multiple models
  • Power BI, Looker, and Tableau Order Entry dashboards are named
  • owners are surfaced so reviewers know who to loop in

There are two ways to use it:

Zero setup path (for judges)
No Docker. No local DataHub. Fork the repo, edit a sample SQL model under demo-pipeline/, open a PR. GitHub Actions runs Tripwire using data/lineage-snapshot.json and comments within seconds.

Live mode (deeper proof)
Run DataHub locally, load showcase-ecommerce, point .env at your instance, and run writeback/writeback.py. This uses the DataHub MCP server mutation tools to stamp tripwire.preMergeRisk onto the touched dataset and related dashboards. You can then open localhost:9002, find Snowflake orders, and see the property under Properties.

That split matters. The snapshot makes judging easy. Live mode proves the snapshot is an export of a real catalog, not invented demo JSON.

How we built it

1. Local DataHub and real demo data

We used DataHub quickstart on Docker, loaded the official showcase-ecommerce datapack, and enabled metadata service auth so we could generate a Personal Access Token. That gave us a real UI at localhost:9002 and GMS at localhost:8080, with Snowflake, dbt, Looker, Tableau, and Power BI entities already connected by lineage.

2. Snapshot exporter

snapshot-exporter/export_snapshot.py connects to live DataHub with the Python SDK, pulls a curated set of order entry entities (orders, customers, order_items, products, order_details), walks downstream lineage up to several hops, and writes data/lineage-snapshot.json.

That file is committed on purpose. It is the portability layer. Judges exercise the same blast radius logic without standing up Kafka, OpenSearch, and MySQL.

3. Bot and GitHub Action

The bot is intentionally small and dependency light:

  • bot/blast_radius.py maps changed paths to snapshot entities and aggregates downstream impact
  • bot/comment_formatter.py turns that structure into a readable PR comment
  • bot/main.py is the Action entrypoint: detect changed demo-pipeline/** files, compute blast radius, post via GitHub's API using GITHUB_TOKEN

.github/workflows/tripwire.yml triggers on pull request open/sync for demo-pipeline/** only. No heavy pip install on the hot path. In practice the check has completed in roughly seven seconds.

4. Live writeback through MCP

writeback/writeback.py reuses the same blast radius logic, ensures the structured property definition exists, then calls the DataHub MCP server tool add_structured_properties with TOOLS_IS_MUTATION_ENABLED=true.

The property value is JSON that includes severity, dashboard count, downstream count, PR URL, timestamp, and source. After a successful run, DataHub shows Tripwire Pre Merge Risk on the entity Properties panel.

5. Packaging for judges

  • README with a two minute try path above the fold
  • examples/ with real sample PR comments

Stack in short: DataHub OSS, DataHub MCP server (mcp-server-datahub), GitHub Actions, Python.

Challenges we ran into

Token auth is off by default in quickstart.
The UI literally said token based authentication was disabled. Without METADATA_SERVICE_AUTH_ENABLED=true on GMS and frontend, we could log into the UI but could not create a PAT or do reliable authenticated MCP mutations. Fixing that was a hard gate for live mode.

Structured property definition vs assignment.
Calling the mutation before the property definition fully existed failed validation. We had to upsert the property definition first (tripwire.preMergeRisk), then assign values onto datasets and dashboards.

Keeping the Action fast.
Judges feel latency immediately. We avoided installing DataHub or MCP inside the PR workflow. Snapshot mode is stdlib Python only. That constraint shaped the architecture more than any fancy abstraction did.

Accomplishments that we're proud of

We have an end to end loop that is boring in the best way:

  1. Edit demo-pipeline/orders_model.sql
  2. Open a PR
  3. Get a real Tripwire comment naming real showcase ecommerce dashboards and owners
  4. Run live writeback
  5. See Tripwire Pre Merge Risk on Snowflake orders in DataHub

We are also proud that writeback is not a slide claim. It is visible in the DataHub UI Properties panel, with a PR link inside the property value.

What we learned

Context that is not in the workflow might as well not exist.
DataHub can be excellent and still fail the merge moment if nobody queries it before approve.

Portability is part of product design.
A live graph is the source of truth. A snapshot is how you let strangers verify you in two minutes. Those are complementary, not contradictory, if you are honest about which path does what.

Writeback changes the story.
A PR comment is ephemeral. A structured property on the entity is institutional memory. For this hackathon especially, contributing back to the graph is the difference between “nice bot” and “agent that does real work.”

What's next for TRIPWIRE

Near term goals:

  • Column level awareness from the PR diff, not only file to table mapping
  • Optional blocking check when severity is high
  • Map DataHub owners to GitHub reviewers more directly
  • Offer writeback as an optional CI step for teams with a reachable DataHub
  • First class config for arbitrary dbt and warehouse repos, so teams do not copy our demo pipeline
  • Support for more change types: drops, type changes, model rename.
  • Explore a small upstream contribution back to DataHub docs or MCP examples around structured property writeback patterns.

Tripwire's bet is simple. The catalog already knows. Ask it before merge, and leave the catalog smarter afterward.

Built With

Share this project:

Updates