ContextForge

Metadata-aware code generation grounded in DataHub. ContextForge analyzes schema, lineage, ownership, governance, and usage before generating impact-aware data code, tests, and review evidence.

Comment

Inspiration

AI coding agents are getting extremely good at producing syntactically valid code, but a data change can be technically correct and still cause real damage.

A simple schema rename can break dashboards, violate governance rules, surprise dataset owners, or silently change downstream business logic.

The missing ingredient is not another prompt. It is organizational context: what exists, what depends on it, who owns it, what rules govern it, and how it is actually used.

DataHub already contains that context.

ContextForge asks: What if DataHub metadata became an active control layer before an AI coding agent writes or ships a data change?

What it does

ContextForge inserts a metadata-aware decision layer before code generation:

DataHub context → impact policy → code + tests → human approval → DataHub decision memory

Before generating a change, ContextForge evaluates five important DataHub context signals:

  • Schema
  • Lineage
  • Ownership
  • Governance
  • Observed usage

It uses that context to determine blast radius and generate three reviewable engineering artifacts:

  • dbt-compatible model / migration SQL
  • schema documentation and tests
  • evidence-linked PR summary

For governed or high-impact changes, ContextForge stops at a human approval boundary.

The AI can prepare the change, but it cannot approve a risky production change by itself.

After approval, ContextForge also includes an explicitly authorized DataHub MCP save_document path that can preserve the approved decision as reusable organizational context for future agents.

How we built it

ContextForge has two intentionally separated execution surfaces.

Judge-ready browser experience

The hosted web application provides a deterministic, credential-free demonstration so judges can immediately test the complete workflow.

It uses a DataHub-shaped context fixture to demonstrate schema, lineage, ownership, governance, and usage-aware generation.

A 14-step Judge Mode walks through the complete experience, automatically demonstrating scenario selection, context analysis, impact policy, generated artifacts, human approval, DataHub write-back preview, and decision memory.

The browser does not pretend that a live DataHub mutation occurred.

Real DataHub MCP integration

The public Python CLI connects to the official DataHub MCP Server.

In live mode, ContextForge can use:

  1. search
  2. get_entities
  3. list_schema_fields
  4. get_lineage
  5. get_dataset_queries

Those metadata signals are normalized into generation constraints and a safety decision.

The write-back path is deliberately more restricted than the read path.

--write-back is rejected in fixture mode and also rejected unless explicit --approve authorization is provided.

Only an approved live execution enables the mutation path and calls DataHub MCP save_document to preserve the decision.

Why DataHub matters

Without DataHub, an AI coding agent primarily sees the requested code change.

With DataHub, the agent can understand the surrounding data system before generating anything.

  • Schema establishes what actually exists.
  • Lineage reveals downstream blast radius.
  • Ownership identifies the humans and teams responsible.
  • Governance carries PII and trust constraints into the change.
  • Usage exposes real business semantics that schema alone cannot show.
  • Decision write-back lets future agents inherit prior approved reasoning and constraints.

That changes the workflow from:

Prompt → code → deploy → discover damage

into:

Understand → assess impact → generate → govern → approve → remember

Challenges we ran into

The hardest challenge was not generating SQL. It was maintaining technical truth while making the value obvious in a short hackathon experience.

We deliberately separated the deterministic browser demo from real DataHub mutation so a polished demo could never be confused with a live write.

We also designed the decision trace as an auditable summary of evidence, impact, policy, and authorized action rather than exposing hidden model reasoning.

Another challenge was making a fairly technical system understandable within minutes. Judge Mode solves that with a guided narrated walkthrough that keeps the workflow focused and easy to evaluate.

Accomplishments that we're proud of

  • Uses five meaningful DataHub context surfaces before code generation
  • Generates three concrete, reviewable engineering artifacts
  • Demonstrates column renames, type migrations, and deprecation scenarios
  • Makes the human approval boundary explicit for governed changes
  • Implements a real approved MCP save_document write-back path in the public CLI
  • Preserves approved decision rationale as reusable context
  • Keeps the browser demo transparent about fixture and local-state behavior
  • Provides a no-login hosted demo
  • Ships a public Apache-2.0 repository
  • Includes a guided 14-step narrated Judge Mode

What we learned

Better prompting alone is not enough for production AI coding agents.

The missing ingredient is organizational context combined with authorization boundaries.

Schema gives an agent syntax-level truth, but lineage, ownership, governance, and usage turn that into operational truth.

DataHub is especially powerful because the same context layer can both inform the decision before generation and preserve the approved outcome afterward.

What's next for ContextForge

The next step is to expose the ContextForge decision and control contract directly to coding-agent workflows so an external agent can request metadata-aware authorization before generating or executing risky changes.

The longer-term opportunity is a reusable context-aware change-control layer for AI agents operating across increasingly complex technical systems.

Built With

Share this project:

Updates