ContextFlow

An autonomous governance agent that audits, fixes, and proves

Comment

Inspiration

A friend who runs a data platform team told me his SOC2 auditor asked for a PII inventory across all their data assets. It took his team three weeks. Three weeks to answer one question. The data was there. DataHub had the lineage and the schemas. But nobody had connected an agent that could walk the graph, find the columns, trace where they went, and write the evidence back.

So I built one.

The first version was terrible. It scanned the catalog and produced a 0/100 health score for every dataset, every time. Not because the catalog was that bad. Because the scoring logic was double-counting. A merged finding with 50 entities was multiplying its penalty 50 times. Classic off-by-one that took two days to find.

The real breakthrough came when I realized the agent needed to say NO. Most AI demos show an agent doing what it's told. A governance agent needs to do the opposite. "Delete all PII columns" should get a refusal, not a success confirmation. The lineage tree showing which dashboards would break is the evidence. The refusal is the product.

What it does

ContextFlow connects to a DataHub instance through the MCP server. It reads schemas, lineage, ownership, and governance metadata across every platform. It finds PII columns, traces them through the entire data stack, fixes documentation gaps on safe assets, and blocks mutations on anything feeding dashboards or ML models.

After every fix, it re-reads the catalog to verify the change is visible in live metadata. A write call returning success is a claim. A re-read is evidence.

It also blocks jailbreak attempts before they reach the model. The injection guard is pure deterministic code no LLM involved, no prompt engineering, no hoping the model behaves.

How I built it

The agent loop uses structured planning before every action (PLAN/GOAL/STEPS/SUCCESS) and metacognition after every tool result ([ASSESS] PASS/BLOCKED/MISSING). The LLM sees enriched tool results through a verification layer that catches empty responses, domain mismatches, and error patterns before the model can hallucinate around them.

The lineage engine queries DataHub GMS directly through GraphQL. I found a bug where Dataset.upstreamLineage doesn't exist on the GMS schema it needs Dataset.lineage(input:) instead. The error was silently swallowed by a catch block, so every blast radius analysis returned "0 downstream consumers" for every dataset. Took three weeks to find that.

The TUI renders in pure ANSI no ncurses, no terminal library, just escape sequences. The thinking indicator is a purple left-marker that shows the model's structured reasoning as it happens. The spinner uses Unicode circle characters in DataHub's brand colors.

Challenges

The hardest problem was making the agent refuse unsafe operations while still being useful. If it says no to everything, it's a glorified grep. If it says yes to everything, it's dangerous. The solution was lineage-gated auto-fix: anything with zero downstream consumers is safe. Anything feeding a dashboard or ML model is held for human review. The agent shows the blast radius tree and asks.

The second hardest was the DataHub connection. The quickstart uses Docker. Auth was enabled by default but the signing key wasn't configured. The MCP server runs separately from GMS. The GraphQL API has different field names than the documentation. Every one of these cost a day of debugging.

The third hardest was making it look good. I killed three npm dependencies (chalk, boxen, ora) and replaced them with zero-dep pure ANSI functions. The spinner, borders, progress bars, and histograms all use raw SGR escape sequences. The binary compiles to 99MB with Bun.

What I learned

  1. A single catch block that silently discards all failure information is ground zero for every P0 bug you will ever have. Fix that first.
  2. Mock-based tests cannot catch API shape mismatches. The add_tags tool used termUrns (from AddTermsInput) instead of tagUrns (from AddTagsInput) for three weeks. All 40+ tests passed because the mock fetch accepts any payload.
  3. Free tier models have real rate limits. Hy3 drained $12 in one day when 5 council subagents all dispatched simultaneously. Switched 6 of 7 roles to free models. Now ~$4.65/month.
  4. MCP tools are reactive. They require the LLM to call them. Text-only models reject images before the tool ever fires. The fix is a plugin hook (experimental.chat.messages.transform) that intercepts images in the message pipeline.
  5. Judges evaluate the demo video, not the code. 680 tests and a 5/5 safety scorecard matter less than 40 seconds of an agent refusing to strip PII tags. Lead with the refusal.

Built With

Share this project:

Updates