Context Hub

AI needs context. Context Hub connects ontologies & data sources via MCP—turning raw data into actionable knowledge for smarter, scalable AI.

Comment

ContextHub

Inspiration

Organizations generate enormous amounts of data, but most of it remains disconnected and difficult to interpret. A service record may contain a team identifier, a repository URL, and a list of dependencies, yet the meaning of those fields often exists only in documentation or in the minds of individual employees.

This becomes an even bigger problem when AI agents work with organizational data. Giving an AI model access to more documents does not automatically give it an understanding of the business. It also needs to know which objects exist, how they relate to each other, and what those relationships mean.

I was inspired by ontology-driven platforms such as Palantir Foundry and graph exploration systems such as GitLab Orbit. However, I wanted users to be able to define their own ontology instead of working with a predefined model.

That idea became ContextHub: a platform that connects freely configurable ontologies with real data and turns them into an explorable, versioned, and AI-ready knowledge graph.

What it does

ContextHub allows users to create multiple independent ontologies inside a workspace.

An ontology can contain:

  • Object types
  • Typed properties
  • Identity and indexed properties
  • Link types with direction and cardinality
  • Interfaces and interface inheritance
  • Shared properties
  • Value types and structs
  • Derived properties
  • Read-only functions

The ontology is created visually with a React Flow editor. Users can connect object types, define relationships, validate their model, and publish immutable ontology versions.

After defining the ontology, users can connect data from:

  • JSON
  • NDJSON
  • CSV
  • Parquet
  • REST APIs
  • GraphQL APIs

ContextHub automatically detects source fields and data types. A visual mapping assistant then connects source fields to ontology properties.

Relationships can also be created directly from source data. For example, an owner_team field can create an owned_by relationship, while a list of service identifiers can create multiple depends_on relationships.

The resulting knowledge graph can be:

  • Searched
  • Filtered by object type
  • Explored in 2D and 3D
  • Navigated through object neighborhoods
  • Queried using a visual Graph Query Builder
  • Inspected for property-level provenance
  • Accessed through gRPC APIs
  • Prepared for read-only AI access through the Model Context Protocol

The visual Query Builder supports bounded filters, projections, sorting, aggregations, and directed graph traversals without exposing raw SQL.

AI integration through MCP

ContextHub is designed to make organizational knowledge available to AI agents through the Model Context Protocol.

Instead of giving an agent isolated documents or unstructured records, ContextHub provides a shared model of:

  • Objects
  • Properties
  • Relationships
  • Ontology definitions
  • Business context
  • Data provenance

The MCP server defines read-only tools for ontology discovery, object search, object lookup, and bounded graph queries.

This gives AI agents the foundation for more accurate answers, better contextual reasoning, and results grounded in the organization’s actual knowledge.

The MCP protocol and tool contracts are implemented. Connecting the MCP tools directly to the persisted graph repository is the next integration step.

How I built it

I implemented ContextHub as a monorepo containing a Next.js frontend, multiple Rust crates, public Protobuf contracts, infrastructure configuration, tests, benchmarks, and a reproducible Devcontainer.

Frontend

The frontend uses:

  • Next.js 16
  • React 19
  • TypeScript
  • Tailwind CSS
  • React Flow
  • React Force Graph 2D
  • React Force Graph 3D
  • ConnectRPC and gRPC-Web

React Flow powers the visual ontology editor. The 2D and 3D explorers use the same graph data model, allowing users to switch between both representations without changing the underlying query.

Backend

The backend uses:

  • Rust
  • Tokio
  • Tonic
  • Prost
  • Axum
  • gRPC and gRPC-Web
  • Serde
  • Tracing

I separated ontology validation, mapping execution, storage, APIs, and MCP into dedicated Rust crates.

Mapping engine

Apache Arrow is the shared in-memory and streaming data format.

Apache DataFusion executes restricted mapping and transformation plans. Users cannot submit arbitrary SQL. Instead, the visual editor creates a declarative MappingPlan that is validated and compiled into safe DataFusion expressions.

Supported transformations include:

  • Rename and cast
  • Trim
  • Uppercase and lowercase
  • String and regular-expression replacement
  • Defaults and coalescing
  • Field concatenation
  • Arithmetic
  • Date and timestamp parsing

Each property mapping can decide whether an invalid value should:

  • Skip the source row
  • Produce a null value
  • Abort the entire import

Storage

ClickHouse stores both the control plane and the knowledge graph.

It contains:

  • Workspaces
  • Ontology drafts
  • Immutable ontology versions
  • Data-source definitions
  • Ontology-specific mappings
  • Import jobs and events
  • Function executions
  • Graph nodes and edges
  • Typed property indexes
  • Provenance information

Graph data is scoped by workspace and ontology version. This allows several ontologies to share data-source definitions while keeping their mappings and graph data completely independent.

MinIO provides local S3-compatible storage for uploaded source files and WASM function artifacts.

Robust imports

Large imports support:

  • Multipart uploads
  • Streaming instead of loading complete files into memory
  • SHA-256 verification
  • Arrow record batches
  • Worker leases
  • Fenced writes
  • Persistent checkpoints
  • Safe retries
  • Stable object and relationship identifiers

Workers can resume interrupted jobs without repeating already completed ClickHouse batches.

Demo

For the demonstration, I created a fictional commerce platform called Nova Commerce.

The source dataset contains:

  • 144 services
  • 8 engineering teams
  • Service ownership information
  • Service dependencies
  • Runtime, environment, region, health, and operational metadata

The imported graph contains:

  • 152 objects
  • 144 owned_by relationships
  • 432 depends_on relationships
  • 576 relationships in total
  • 0 rejected records

The demo shows the complete journey from ontology creation and data mapping to import history, provenance, visual graph queries, 2D exploration, 3D navigation, and AI-ready context through MCP.

Challenges I faced

Supporting arbitrary ontologies

The platform cannot assume that every user has objects such as Service or Team. Object types, properties, interfaces, and links must be validated and processed dynamically.

I solved this by separating ontology definitions from graph data and validating every mapping and query against an immutable ontology version.

Using ClickHouse as a property-graph store

ClickHouse is not a traditional graph database. I needed to design stable node and edge tables, typed property indexes, bounded traversals, keyset pagination, and parameterized query compilation.

This required careful query planning and strict graph-query limits, but it also provided a single high-performance system for metadata, analytics, indexes, and graph storage.

Mapping several object types from one source

A single source record can produce multiple objects and relationships. A service record, for example, may create both a Service and a Team object.

I designed the ingestion worker to execute multiple object mappings over the same Arrow batches, merge objects globally, and resolve relationships only after all target identities are available.

Streaming formatted JSON

Small one-line JSON files worked correctly, while realistic formatted JSON arrays initially failed during Arrow ingestion.

The issue was caused by passing multi-line objects into a line-oriented JSON reader. I fixed the streaming normalizer so every parsed object is serialized into a valid compact NDJSON record. I also added a regression test for pretty-printed JSON containing nested arrays and objects.

Keeping ontology versions synchronized

An import publishes a new immutable ontology version. Initially, the Explorer could display the new graph while provenance requests still used the previous version identifier.

I fixed the import workflow so the frontend immediately records the newly published ontology version before opening the graph.

Visualizing dense graphs

Showing hundreds or thousands of labeled objects creates visual and performance challenges.

I introduced object limits, type filters, focused neighborhoods, incremental expansion, search, camera controls, and different budgets for 2D and 3D rendering.

Making imports resumable

Large imports must survive backend restarts and multiple worker replicas.

I implemented expiring worker leases, fencing tokens, stable identifiers, bounded ClickHouse batches, and persistent node and edge checkpoints.

What I learned

One of my biggest lessons was that an ontology is more than a schema. It must connect definitions, data mappings, validation, versioning, graph queries, provenance, visualization, and AI access.

I also learned that provenance needs to be designed into the ingestion pipeline from the beginning. Recording only the final property value is not enough. Users need to understand where the value came from, which mapping produced it, and which import job wrote it.

Apache Arrow proved to be a strong common boundary between very different data sources. Once a connector produces Arrow record batches, the rest of the mapping and ingestion pipeline can remain largely source-independent.

I also learned how to use ClickHouse as both a control plane and a bounded property-graph store. This required a different approach from using a traditional graph database, especially for traversals, property indexes, versioning, and pagination.

Finally, I learned that AI integration becomes much more useful when a model receives structured relationships and explicit semantics instead of simply receiving more unstructured text.

Accomplishments that I am proud of

  • I built the project as a solo developer.
  • Users can create multiple freely configurable ontologies.
  • Data sources can be shared without sharing ontology-specific interpretations.
  • One source can create multiple object types and relationships.
  • JSON, NDJSON, CSV, Parquet, REST, and GraphQL sources use one mapping workflow.
  • Imports are streamed, resumable, and idempotent.
  • The graph can be queried without exposing raw SQL.
  • Properties retain traceable source provenance.
  • The same graph can be explored interactively in 2D and 3D.
  • Functions can run as controlled expressions, external gRPC providers, or sandboxed WASM modules.
  • The project includes Playwright end-to-end tests.
  • The project includes a benchmark for one million nodes and five million edges.
  • The system provides the foundation for ontology-aware AI agents through MCP.

Built With

  • clickhouse
  • datafusion
  • graphql
  • mcp
  • nextjs
  • ontology
  • rest
  • rust
Share this project:

Updates