Traversa - Knowledge Base

Traversa - Knowledge base

Inspiration

It started with an idea Andrej Karpathy came up with: "LLM wiki", an agent that compiles raw documents into a linked, structured knowledge base once, instead of retrieving from scratch on every query. Karpathy built a rough version of it himself, but shared it as simple scripts rather than a real system an idea meant for others to pick up and build out. So we're giving it a try.

What it does

Instead of retrieving raw chunks, Traversa compiles what it ingests; pasted text, links, or files into a linked, deduplicated knowledge graph on CockroachDB. Every query starts with a vector search into the graph, then traverses connected concepts to build an answer, flagging contradictions across sources rather than silently resolving them.

This approach have several benefits, namely:

Multi-hop reasoning — graph traversal surfaces related concepts a single vector match would miss, unlike flat chunk retrieval
Contradiction detection — conflicting information across sources gets flagged explicitly instead of being silently averaged or arbitrarily picked by similarity score
Faster queries at scale — traversing a pre-compiled graph is cheaper than re-embedding and re-ranking raw text every time
Auditable knowledge — you can inspect exactly which sources fed a concept and how concepts connect, unlike black-box chunk retrieval -Shared memory between user and agent — because the agent and user work from the same knowledge graph, users can inspect, correct, and expand the knowledge base while the agent benefits from those updates in future queries.

How we built it

We started with the compile pipeline. When a user saves a URL, PDF, or note, we create a raw item and put a compile job into an arq queue backed by Redis. A worker picks up the job and passes it to our agent, which is built with Mastra. The agent fetches the content, uses Amazon Bedrock Mantle to extract claims and suggest relationships between them, and then writes the results back through our API. Wiki pages, sections, claims, nodes, and edges are all saved to CockroachDB in a single transaction. We also use Louvain community detection to group related concepts, while a custom force-directed layout handles the graph visualization on the client.

Some of the architecture came from problems we ran into along the way. We decided not to run the agent as a serverless function because a compile job can involve several rounds of reasoning and tool calls, so it made more sense to keep it as a long-running process. We also use two ORMs: SQLAlchemy for writes on the Python API and Prisma for reads on the TypeScript side. Redis is used for both the job queue and real-time updates, which we stream to the client through SSE.

Authentication is kept separate from the rest of the system. Better Auth issues JWTs, and our other services verify them through JWKS without directly accessing the auth database. For the database, Prisma also doesn't support defining pgvector indexes in its schema, so we add those indexes manually in our migrations. Finally, the whole stack, including the client, API, agent, and worker, is containerized with Docker and runs on a single EC2 instance with Github Action CI/CD.

Challenges we ran into

The hardest problems weren't really in the infrastructure. They were in getting the agent to make the right decisions. Compiling isn't just summarizing a document. The agent has to figure out what should become a new concept, what is a duplicate of something already in the wiki, and how different concepts should be connected. Early on, it would create multiple nodes for basically the same idea, or completely miss connections between pages that were clearly related when you looked at them yourself. We went through several iterations of the prompt and changed what context we gave the agent during retrieval before the extraction and deduplication became reliable enough.

Once the wiki started getting larger, we ran into another problem with the graph visualization. With only a few dozen concepts, the force-directed layout started getting messy. Nodes would jitter around, edges would overlap, and it became hard to tell what was connected to what. We ended up tuning the physics simulation and making use of the Louvain clustering we already had on the backend. We use those clusters to group related nodes visually, which keeps the graph readable as more concepts are added.

Accomplishments that we're proud of

If we had to pick one thing we're most proud of, it's the contradiction detection. It's easy to build a system that just replaces old information with new information, or picks the source that seems more confident and moves on. We wanted Traversa to do something different. When two sources actually disagree, the agent should recognize that and show the disagreement to the user instead of quietly choosing one. Getting that to work reliably was probably the hardest part of the original idea, and also the part we cared most about.

We're also proud of how much we managed to build in such a short time. We went from a rough idea inspired by a tweet and a few scripts to a working system with ingestion, compilation, graph storage, traversal, and visualization, all running on real infrastructure. We could have made the demo look convincing with static data behind the UI, but everything shown is actually being compiled by the agent in real time.

Another thing we cared about was making the write path reliable. No matter how many claims or edges a compile job produces, everything gets written to CockroachDB as one transaction. If something fails, we don't end up with half of a graph written to the database. It's a small implementation detail, but it matters when the whole system depends on the graph staying consistent.

What we learned

Going in, we thought the hardest part of "compile, don't retrieve" would be the compilation itself, basically extracting claims and concepts from a document. It turned out that wasn't the difficult part. The difficult part was merging new information into what we already had. The agent had to decide whether a new claim was actually new, just a different way of saying something we already knew, or a contradiction. Most of our iteration during the hackathon ended up going into that problem rather than the extraction itself.

That also taught us something about working from a rough idea. Something being presented as a simple gist doesn't mean it's simple to build. It usually means the difficult implementation details are left for whoever decides to take the idea further. Karpathy's write-up explains the core concept in a few paragraphs, but getting compilation, deduplication, and traversal to work together reliably took us the entire hackathon.

If we had to sum up the whole hackathon in one sentence, it would be this: retrieval was never really the bottleneck for agent memory. Understanding was. Finding a relevant chunk is relatively easy. Deciding what is new, what is redundant, and what conflicts with what we already know turned out to be the much harder problem.

What's next for Traversa - Knowledge Base

For the next version of Traversa, we want to focus on three areas: customization, specific use cases, and better agent capabilities.

First, we want users to have more control over how their knowledge base is built. Different users will want to organize information differently, so we'd like to let them define their own categories, relationships, sources, and rules for how information is compiled. The goal is to make the wiki feel less like a fixed system and more like a knowledge base that can adapt to how someone actually works.

We also want to explore more specific use cases instead of trying to make Traversa a general-purpose knowledge base. For example, it could be used for research, where someone continuously adds papers and tracks how different findings relate or contradict each other. It could also work for technical documentation, project knowledge, or keeping track of a specific domain over time. Building around these use cases would let us make the compilation and retrieval process much more useful for each one.

Finally, we want to improve the agent itself. Right now, the agent is mainly responsible for extracting and connecting knowledge. We want it to become better at reasoning over the knowledge base, identifying gaps, deciding when more information is needed, and using the existing graph to form more complex answers. The long-term goal is for Traversa to not just remember what you've given it, but actually become better at working with that knowledge as the wiki grows.