-
-
Ask a question, get a cited answer. A second Nemotron model checks every claim against the sources and marks it Verified.
-
Before any model sees the text, personal data becomes placeholders. You compare original and redacted, then approve.
-
Ask for the SSN and the model only sees [SSN_1]. Ask for something not in the document and it says it doesn’t know.
-
Wire Inspector: the exact payload sent to the model. Names, SSNs and emails appear only as placeholders.
-
The Privacy tab counts every value hidden, per document, and lists every model call.
-
Hash-chained audit log: each entry includes the previous hash, so any edit breaks the chain. Verify shows it intact.
-
valuation: 13/13 questions passed and the Ultra judge flagged 7/7 deliberately wrong answers. Small synthetic test.
-
After approval the document is ready: values hidden, passages indexed, semantic search ready.
-
The Chat tab with suggested questions. Your questions are redacted too, before they are sent.
-
Purge clears documents, embeddings, chat, wire log and audit log from the browser, and returns a receipt.
-
After a purge there is nothing left to ask about. Session data only ever lived in the browser.
-
VaultAI start screen: drop a .txt or .pdf, or load the synthetic sample. Only redacted text reaches a model.
Inspiration
Teams in healthcare, finance and legal want to use AI on their documents, but they cannot paste patient records or loan files into a system they cannot inspect. Most "private AI" tools just ask you to trust them. We wanted one that lets you verify.
What it does
VaultAI redacts personal data (names, SSNs, emails, phone numbers, addresses, IBANs, card numbers, dates and labelled IDs) on our server before any model sees the text, and shows a side-by-side preview for approval. Questions are answered by NVIDIA Nemotron Super with numbered citations. A second model, Nemotron Ultra, acts as an independent judge: it splits each answer into claims and marks each as supported or unsupported by the sources. If the first check scores low, the answer is regenerated once with a stricter prompt and re-checked. User questions are redacted too.
You can also inspect everything: a Wire Inspector shows the exact payload sent to the model, a hash-chained audit log records every action and can be integrity-checked, and a Purge button clears the session and returns a receipt.
How we built it
Next.js and TypeScript, deployed on Vercel. Everything model-related runs through Nebius Token Factory using the standard openai package with a changed base URL. We route each task to the smallest suitable model:
Nemotron Nano for query cleanup Nemotron Super for grounded, cited answers Nemotron Ultra as the judge Qwen3-Embedding-8B for embeddings (the only embedding model listed on Token Factory)
Session data (chunks, vectors, audit log) lives in the browser, so the server is stateless and Purge means something real. Retrieval combines vector search with BM25 keyword search and falls back to keywords if embedding is slow. A shared prompt file keeps the app and the evaluation measuring the same thing. There is also an offline mock mode, 18 unit tests and 12 API smoke checks.
Challenges we ran into
Reasoning models spend max_tokens on thinking, which produced empty answers and truncated judge output until we raised limits and added a retry. Embedding latency varied from under a second to over ten, so we added keyword search and a timeout fallback. Judge output sometimes arrived as malformed JSON, and the models sometimes cited with fullwidth brackets. We validate, retry and normalise both. Our first evaluation prompts had drifted from the app's prompts, so the numbers did not describe the app. We moved every prompt into one shared file.
Accomplishments that we're proud of
In our final evaluation (Oct 29), all 13 questions passed, including a document with an embedded prompt-injection instruction (ignored), a question that needs two documents, and an amendment that overrides an earlier value. The Ultra judge flagged all 7 deliberately wrong answers, including a dose with the wrong frequency and a previous lab value presented as current. A full evaluation run costs about $0.058. We report this as a small synthetic smoke test, not a benchmark.
What we learned
Verification is more useful than a confidence score: showing which claim is unsupported changes how much you trust the rest. We also learned that model "reasoning" output needs generous token budgets, and that you should measure the app you ship, not a parallel copy of it.
What's next for VaultAI
A stronger redaction model for names in plain prose, OCR for scanned PDFs, an NVIDIA embedding model when one is available on Token Factory, and larger, real-world evaluations
Honest limits
Redaction is rule-based and misses unlabelled names; there is no OCR; the judge is a model and can be wrong; the demo data is synthetic. These are also listed in the README.
Built With
- bm25
- nebius-token-factory
- nemotron-3-nano
- nemotron-3-super
- nemotron-3-ultra
- next.js
- node.js
- nvidia-nemotron
- openai-sdk
- pdf-parsing
- qwen3-embedding
- rag
- react
- tailwindcss
- typescript
- vercel
- web-cypto
- zod