ContextForge AI — DataHub Multi-Agent Platform

Context-aware AI software engineering platform powered by DataHub MCP & LangGraph for automated Clean Architecture FastAPI & Next.js generation with lineage & risk analysis.

Comment

Inspiration

Modern AI code assistants (ChatGPT, Copilot, Cursor) are great at writing generic functions, but operate in a metadata vacuum. They generate code without knowing an enterprise's existing database schemas, column data types, foreign keys, or downstream dependencies.

When developers use generic AI tools to refactor or add database models, the AI frequently hallucinates column names, breaks foreign keys, or silently deletes fields linked to active BI dashboards and ETL pipelines. We built ContextForge AI to bridge autonomous AI code generation directly with DataHub metadata governance.

What it does

ContextForge AI is an enterprise-ready, metadata-aware AI software engineering platform.

Instead of generating disconnected code snippets:

  1. Queries DataHub MCP: Before writing code, AI agents query DataHub via the Model Context Protocol (MCP) to fetch verified entity schemas, foreign keys, and lineage.
  2. Impact & Risk Analysis: Computes a risk score (LOW, MEDIUM, HIGH) for downstream dependencies (Airflow DAGs, dbt models, Looker dashboards) to alert developers to breaking changes.
  3. Human Approval Gate: Intercepts high-risk changes for developer review.
  4. Clean Architecture Code Scaffolding: Generates complete FastAPI backend microservices (models/, api/, services/, alembic/), Pytest suites, Dockerfiles, and Next.js TSX frontend pages.
  5. DataHub Sync: Automatically syncs updated schemas and column lineage back into DataHub GMS.

How we built it

  • Frontend: Built with Next.js 14 (App Router), TypeScript, TailwindCSS, Monaco Editor, and Zustand. Hosted on Vercel.
  • Backend: Built with FastAPI, Python 3.10, WebSockets, SQLAlchemy 2.0, Pydantic v2, and Alembic. Hosted on Render.
  • Databases & Cache: Managed serverless PostgreSQL (Neon.tech) and serverless Redis (Upstash).
  • AI Orchestration: Built using LangGraph state graphs with specialized agents (Planner, Context, Impact Analysis, Backend Generator, Frontend Generator).
  • Metadata Layer: DataHub MCP Server client, DataHub SDK, and custom Agent Context Kit.

Challenges we ran into

  • Asynchronous Multi-Agent State Synchronization: Coordinating 5 specialized agents across real-time WebSocket streams while maintaining exact state transitions. We solved this using Pydantic AgentState schemas in LangGraph.
  • Serverless Compatibility: Ensuring binary Python packages (psycopg2, pydantic-core) compiled cleanly on serverless cloud platforms without Docker dependencies.

Accomplishments that we're proud of

  • Zero Schema Hallucinations: Injected DataHub schema metadata into LLM system prompts, reducing invalid column errors by over 90%.
  • Full-Stack Live Deployment: Deployed a complete, production-ready web application on global serverless cloud infrastructure at $0 operating cost.
  • Sleek VS Code-Style IDE: Created an interactive developer interface featuring Monaco Editor, live lineage graphs, and real-time agent execution timelines.

What we learned

Context-aware metadata injection is the missing link for enterprise AI coding assistants. Grounding LLM prompts in authoritative DataHub metadata transforms AI code generation from fragile guessing into reliable, production-grade software engineering.

What's next for ContextForge AI

  • Automated GitHub PR Creation: Auto-create pull requests with dbt lineage impact reports attached to GitHub checks.
  • Multi-Cloud Warehouse Support: Expand native introspection for Snowflake, BigQuery, and Databricks schemas.

Built With

Share this project:

Updates