Levee

A pre-commit dbt lineage guardian that intercepts GitHub PRs, traces downstream impact through DataHub via MCP, and blocks merges that would silently break production ML models.

Comment

The Problem

A dbt model gets a column renamed. The PR looks clean — tests pass, lint is green, CI is green. It merges. Three days later, the executive churn dashboard shows null. The ML team's feature pipeline has been ingesting wrong values since the merge. Nobody connects the cause to the change because three days passed.

CI catches code errors. DataHub knows the lineage. But nobody puts the lineage graph in the merge path automatically. Levee does.

What Levee Does

Levee intercepts GitHub pull requests that modify dbt models, traces their full downstream impact through DataHub's lineage graph via mcp-server-datahub, and issues a verdict before the PR can merge:

  • BLOCK — a PROD-tagged asset or production ML model is downstream. The PR is blocked. Levee opens a companion fix PR with a safe migration already committed.
  • WARN — downstream dependents exist but none are production-critical. Levee posts an impact report and lets the engineer decide.
  • APPROVE — no downstream dependents. Merge safely.

Every verdict is written back to DataHub — the blocked asset is tagged urn:li:tag:levee-blocked and an Incident aspect is created with the full impact report, so the next agent or engineer who touches the asset inherits the context.

The Companion Fix PR

When Levee blocks a PR, it doesn't just post a comment. It creates a new branch with the generated safe migration already applied, opens a PR against the original branch, and links it in the block comment. The engineer merges the fix PR first, then their original PR passes.

Zero manual SQL copying. The fix is a mergeable artifact, not a suggestion in a text box.

How We Built It

  • GitHub OAuth App: real browser-redirect OAuth flow. Any user clicks "Connect Your Repo," authenticates with GitHub, selects their repo, and Levee registers the webhook automatically. Personal workspaces are isolated per connected repo.
  • Webhook receiver: HMAC SHA256 signature verification on every incoming event. Unverified payloads rejected with 401.
  • Diff parser: extracts column renames, drops, and type changes from real git patch diffs into structured change manifests
  • Lineage tracer: mcp-server-datahub downstream traversal up to 6 hops, classifying each asset as CRITICAL / HIGH / MEDIUM / LOW
  • Impact scorer: BLOCK / WARN / APPROVE verdict engine with fails-closed behavior — DataHub unreachable means BLOCK, never silent APPROVE
  • Migration generator: produces safe dbt SQL for rename and drop scenarios — COALESCE aliases, backfill queries, ref() update lists
  • DataHub write-back: acryl-datahub SDK for tagging assets and writing Incident aspects
  • Frontend: Next.js 16.2 PR review queue with live polling, personal workspace per connected repo, in-UI fix PR merge action

The Real User Flow

  1. Engineer opens a dbt PR renaming billing_amount → amount
  2. GitHub sends webhook to Levee
  3. Levee traces lineage via DataHub MCP, finds billing_risk_model [PRODUCTION ML] three hops downstream
  4. Levee posts BLOCK comment on the PR with consequence list and fix PR
  5. Levee opens levee/fix-pr-104 with the safe COALESCE migration
  6. Engineer merges the fix PR
  7. Engineer re-opens their original PR — it now passes
  8. Levee posts APPROVE, tags the asset levee-approved in DataHub

Challenges

  • GitHub OAuth state validation had to handle the case where either the cookie or the query parameter is missing — not just when they disagree. Both must be present and match.
  • Vercel's serverless functions cannot run long-lived Python MCP subprocesses or write durable file-based state. The architecture requires a persistent Python backend; for the demo this runs locally with the Next.js frontend deployed separately.
  • Column-level lineage in DataHub is sparse in practice. Levee uses dataset-level lineage with asset classification rather than relying on column-to-column edges, and documents this accurately rather than overstating the precision of the impact detection.
  • The companion fix PR flow required careful sequencing: create branch, commit migration, open PR, post comment on original PR, write back to DataHub — all in order, all with explicit failure handling at each step so a missing token produces a clear error rather than a fabricated URL.

Built With

DataHub · mcp-server-datahub · acryl-datahub SDK · GitHub OAuth Apps · GitHub Webhooks API · Next.js 16.2 · TypeScript · Tailwind CSS · Python 3.11 · Claude (claude-cli)

Built With

Share this project:

Updates