ContextForge

Metadata-Aware Code Generation & Development challenge

Comment

Inspiration

AI agents are becoming increasingly capable, but they still struggle when working with organizational data. A general-purpose model may know how to write SQL or create a dbt model, yet it often lacks the context needed to produce code that is safe and production-ready. It may invent columns, misunderstand table relationships, ignore governance requirements, or fail to identify downstream systems affected by a change.

I built ContextForge to solve that problem.

My goal was to create a development agent that does not generate data code from a prompt alone. Instead, it first retrieves trusted context from DataHub, including schemas, field descriptions, ownership, tags, glossary terms, and lineage. It then uses that context to generate artifacts that a data team could realistically review and merge.

ContextForge demonstrates what becomes possible when AI agents understand the data environment before taking action.

What it does

ContextForge is a metadata-aware code generation application powered by DataHub.

A user selects a dataset from DataHub and asks ContextForge to generate a transformation. The application retrieves the dataset’s metadata and uses it to produce a PR-ready bundle containing:

  • A dbt SQL model based only on verified columns
  • A dbt schema YAML file with descriptions and suggested tests
  • A change-impact report based on upstream and downstream lineage
  • Ownership and governance checks
  • Warnings for missing descriptions, owners, or business context
  • A downloadable ZIP containing the generated artifacts

Before generating code, ContextForge validates the available schema so that it does not reference nonexistent fields. It also surfaces relevant governance information, such as tags and glossary terms, and highlights downstream assets that may be affected by the proposed change.

The application can run with built-in sample metadata for demonstrations or connect to a local DataHub instance through its API.

How we built it

I built ContextForge as a Python application with a Streamlit user interface.

DataHub acts as the context layer. The application queries DataHub for metadata associated with a selected dataset, including:

  • Dataset and platform information
  • Field names and native data types
  • Field-level descriptions
  • Ownership information
  • Tags and glossary terms
  • Upstream and downstream lineage

I created a structured internal representation of this metadata so that the generation layer receives consistent, validated context.

The code generation workflow has three main stages:

  1. Context retrieval: ContextForge retrieves metadata from DataHub or loads the included demonstration dataset.
  2. Validation and generation: The application verifies available fields and generates SQL, dbt YAML, tests, and documentation using the retrieved context.
  3. Artifact packaging: The generated files are displayed in the interface and packaged into a downloadable ZIP that can be added to a Git repository or pull request.

I also added deterministic generation logic so the project can be demonstrated without requiring a paid model API. The architecture includes an extension point where an LLM can later be added for more advanced natural-language planning and code generation.

The repository includes:

  • A Streamlit application
  • DataHub client and metadata models
  • Code-generation utilities
  • Sample generated artifacts
  • Automated tests
  • Local setup instructions
  • An Apache 2.0 license
  • A Devpost submission description
  • A three-minute demonstration script ## Challenges we ran into One of the main challenges I faced was converting DataHub’s rich metadata into a compact structure that could reliably guide code generation. Metadata can be incomplete, inconsistent, or spread across schemas, ownership records, tags, glossary terms, and lineage relationships. I needed to normalize this information before it could be used safely.

Another challenge was preventing hallucinated columns. A code generator can easily produce syntactically correct SQL that references fields that do not exist. I addressed this by validating all generated column references against the schema retrieved from DataHub.

Lineage analysis was also challenging because downstream impact is not always obvious from the selected dataset alone. I designed the change report to surface known upstream and downstream dependencies while clearly distinguishing verified metadata from recommendations.

I also wanted the project to be easy for judges to evaluate. Requiring external API keys or a complex hosted environment could make the demonstration difficult to reproduce. To solve this, I included a deterministic demo mode, sample outputs, and a local DataHub setup path.

Finally, balancing useful automation with human review was important. ContextForge generates draft artifacts and risk warnings, but it does not claim that every generated change should be merged automatically. The generated bundle is designed to support a pull-request workflow where a data engineer remains in control.

Accomplishments that we're proud of

I am proud that ContextForge produces more than a conversational answer. It creates concrete development artifacts that can be reviewed, downloaded, committed to a repository, and included in a pull request.

I am especially proud of the following:

Generated SQL only uses fields verified through DataHub metadata. The application includes lineage-aware change analysis. Governance context is incorporated into the generated output. Missing ownership and documentation are treated as visible risks rather than ignored. The project works without requiring a paid LLM service. Sample outputs are included so judges can evaluate artifact quality immediately. The complete project is open source under the Apache 2.0 license. The architecture can support future MCP Server or Agent Context Kit integrations.

ContextForge shows how DataHub can move an agent from generic code generation toward context-aware data engineering.

What we learned

I learned that metadata is not simply documentation for humans. It can function as operational context for AI systems.

Schema information helps an agent generate valid code. Lineage helps it reason about impact. Ownership identifies who should review a change. Glossary terms provide business meaning. Tags and governance metadata help the agent recognize sensitive or controlled assets.

I also learned that grounded generation requires more than adding metadata to a prompt. The metadata must be normalized, validated, prioritized, and connected to specific actions in the development workflow.

Another important lesson was that an effective agent should communicate uncertainty. When ownership, documentation, or lineage is missing, the agent should not silently assume the answer. It should surface the missing context and explain how it affects confidence in the generated artifact.

Most importantly, I learned that DataHub can act as a shared memory layer between humans, data systems, and AI agents. An agent can read existing organizational knowledge, generate an artifact, and eventually write new knowledge back so that future users and agents inherit better context.

What's next for ContextForge

My next step is to integrate ContextForge more deeply with the DataHub MCP Server and Agent Context Kit so that agents can retrieve context through standardized tools and workflows.

I also plan to add:

LLM-powered natural-language transformation planning Automatic creation of Git branches and pull requests Support for Airflow, Dagster, and Prefect pipeline generation Generation of ingestion recipes and migration scripts Column-level lineage analysis Policy-aware checks for sensitive data Data contract generation and validation Integration with CI pipelines Automated SQL linting and execution testing Writing generated documentation and decisions back into DataHub Multi-agent workflows for generation, validation, governance review, and approval

My long-term vision is for ContextForge to become a metadata-aware engineering teammate: an agent that understands an organization’s data stack, produces implementation-ready changes, explains their impact, and preserves the resulting knowledge for the next developer or agent.

Built With

Share this project:

Updates