Corpus Wiki

The Self-maintaining Knowledge Space. Docs go stale the day they are written. But with Corpus Wiki along with NVIDIA Nemotron the knowledge updates itself and can create things more out of it.

Comment

Inspiration

All companies that I have been employed in- they always seems to have the same problem... No one really documents anything and if they did usually that documentation is old and really need revision or updates. Also, in my personal life, I like to record my thoughts so I have lot of audio files on my phone. But, thought of organizing all of that into something useful feels like a tall mountain to climb. And not mention the amount of random stray note I have and files of I don't know anymores. For the longest time I wished something existed that could just maintain all that information for me. Packaged up in a neat organized and up to dated manner.

A lot of solutions came up through the years like Notions, Obsidian and even chatgpt. But all of these technologies came with some form of maintenance associated with them. Ex. Notion {formats}, Obsidian {brackets} and Chatgpt ask it summaries and it just makes you another document to store.

What I really wanted was something where I can just drop a document or anything and it would just magically sort it self out somehow with me doing that work. But, with the rise of agents and these automated and long running workflow. I felt it's App that is possible now. But to my disappointment no one really has built it yet. Sure, there's applications that handle memory, context or even some second brains people are building. But none of them really does what I imagine a product like that should look like. So, using this hackathon as motivation I decided to build it myself.

What it does

At a basic level. Corpus Wiki will scan any form of information like docs, video, images and whatever. And it will split it up into chunks called Knowledge Units which is much easier to handle than PDF Doc. Then using these knowledge unit plus an LLM plus an embedding space for fast queries. In theory, you can transform that knowledge space into any type of information. Video, Text, Wiki, PowerPoints, and etc. It might sound like RAG with extra step. But the difference here is the pre-classification step we do in this process we try to keep the connections, domain, ontology and meaning of these knowledge chunks while maintaining that information's uniqueness. Where, a single mutable claim uniquely makes sense on its own in its domain.

FEATURES

  • Upload anything. PDF, Word, Markdown, notes, images and video. It reads the pictures too.
  • Short facts with sources. Each file gets broken into small facts, and each fact keeps its source.
  • The AI drafts, the database decides. Nemotron drafts a ChangeSet (add, change, merge or retire facts). A Postgres function checks it and applies it in one go.
  • It asks when it's not sure. Two sources disagree, something gets deleted or merged, it waits for a person. Safe stuff just goes through.
  • Pages rewrite themselves. When a fact changes, only the lines that use it get rewritten, then fact checked.
  • Undo anything. Every change goes in a change log with its cause and an undo.
  • More than pages. A glossary, a stakeholder map with power levels, quizzes with spaced review, how-to docs and narrated videos, all from the same facts.
  • Memory for your agents. It's an MCP server, so Claude, Cursor or ChatGPT can read and write the same wiki. Same rules as people.

How we built it

I started on October 14. During Tech Week, in SF. I joined on of those Builder & Brews event. I had fun. After I flew back, I started working on it some more. It's around 577 commits now. Most of the code was written with Claude Code agents running in parallel, each one on its own piece (change control, uploads, living pages, screens), and I merged them together. I'd write a plan and a check first, then let the agents go until the check passed. The biggest hurtle here designing a good architecture leveraging my knowledge base to build a robust but also easy to use system. I used a standard web stack with React, Next.js, Supabase while using Nebius's product suite. I use token Factory with custom skills for a fleet of 20 Agents for the workflow and evals. Also, I use Nebius cloud for long running task such as video generation etc.

SPECS App: Next.js 16 and React 19 on Vercel. Supabase for Postgres, pgvector, Realtime, storage and login.

AI: every AI step runs on open NVIDIA Nemotron models through Nebius Token Factory. One API key.

  • Nemotron 3 Ultra for the big calls: planning articles, the topic tree, page structure
  • Nemotron 3 Super for the daily work: reading uploads, writing lines, fact checking
  • Nemotron 3 Nano for bulk linking
  • Nemotron 3.5 Super VL for images, diagrams and video frames
  • Qwen3 Embedding 8B to find facts that say the same thing in different words

Video: narrated videos render as Nebius Serverless AI jobs on a CPU VM (headless Chromium, HyperFrames, ffmpeg). You only pay while it runs.

Challenges we ran into

  • Not letting the AI write knowledge directly. The model only drafts. The database checks every action. Each ChangeSet saves which model and prompt made it, so a bad batch can be traced and undone.
  • Rewriting whole pages. It was slow, expensive and lost edits. So every line became its own row that cites the exact facts it says. Now only the stale lines get rewritten.
  • Testing out OrioleDB. Supabase runs on OrioleDB. A GIN index on an array column made inserts drop the connection and hid rows. Took a while to find. Now there's a rule: no GIN indexes.
  • Search got worse before it got better. I switched to hybrid search thinking it'd help and hit@5 dropped from 90% to 53%. Adding a reranker took it to 100%. Lesson: measure before you switch.
  • Video doesn't fit in a web request. It needs a browser, ffmpeg and minutes of CPU, so I moved it to Nebius jobs. CPU quota was 0 at first, and the callback needed a public URL, so now the app just polls Nebius for the result.
  • Cost over run from loops. The reviewer role was firing way more than it should. I built a cost report per role and per model to find it, then batched fact checks per section.

Accomplishments that we're proud of

  • It works end to end. Upload a file and the page updates only the lines that changed, with sources.
  • Every line traces back to a file and a page.
  • Risky changes wait for a person. Everything else just happens.
  • A full MCP server with OAuth 2.1 and 50+ tools.
  • Built in about two weeks.

What we learned

  • Smaller models do most jobs fine. Save the big one for the hard calls
  • Measure before you turn something on
  • Use your own UI library
  • Creating agent workflow is tricky and will bankrupt you if you don't watch it.

What's next for Corpus Wiki

  • If people like I will turn it to real product
  • Use it of my personal docs
  • Build out more features and optimize the work flow so it's fast, cheap and even more reliable. Something companies will use.

Built With

Share this project:

Updates

Submission history