Inspiration
Our most useful personal information is scattered across medical PDFs, insurance emails, WhatsApp chats, scans, and Office files. AI can connect those dots, but asking a cloud model usually means uploading raw records, identifiers, and secrets.
I built VaultAgent to make that trade unnecessary: useful answers without surrendering the underlying personal data.
What it does
VaultAgent is a privacy-first personal vault for macOS.
It indexes PDFs, Office documents, text/Markdown, email archives, WhatsApp exports, and image scans locally. When a user asks a question such as “What did my cardiologist prescribe, and did insurance cover it?”, retrieval happens on-device first.
Before any context reaches a reasoning model, VaultAgent passes it through a local redaction gate:
- Microsoft Presidio catches structured PII such as cards, Aadhaar/Social-Security, PAN, phone numbers, emails, OTPs, passwords, and API keys.
- Gemma 4 E2B checks contextual/prose PII locally.
- Detected values become typed placeholders such as
[CREDIT_CARD].
The dashboard shows the exact local text beside the redacted payload that left the Mac, so privacy is inspectable rather than a promise.
How I built it
The local stack uses Apple Vision OCR, EmbeddingGemma, SQLite with FTS5 and sqlite-vec, FastAPI, Ollama, Presidio, and FastMCP.
GPT-5.6 runs through the Codex runtime only after the redaction gate. VaultAgent also exposes a tunnel-friendly read-only MCP for ChatGPT and a separate localhost-only MCP for trusted local Codex file imports.
I used Codex as an engineering collaborator to build the evaluation harness, reasoning evaluation, audit leak sweep, and test-first module comparisons.
Evidence
- 525/525 hard redaction stress cases clean.
- On contextual/prose PII, the Full gate measured 0.533 recall versus 0.213 for Presidio alone.
- A real-audit replay checked 1,786 redacted fields against 8 canaries: 0 leaks.
- GPT-5.6 Terra answered 9/10 fixed vault questions; Luna answered 8/10.
- 44/44 regression tests pass.
Challenges
The hard part was not retrieval, it was making privacy behavior reliable under OCR noise, long documents, cached sends, edits, deletions, and cancellation. I added deletion-aware indexing, per-file removal/reset controls, authenticated MCP boundaries, opaque record IDs, and an audit trail so the system can be checked after every request.
What I learned
Privacy UX has to be visible. Users should not need to trust a badge saying “secure”; they should be able to see what stayed local, what was masked, and what was sent. I also learned that speed and privacy are a real trade-off, so VaultAgent exposes Fast and Full modes honestly instead of hiding the difference. I also learned that Speed matters.
Log in or sign up for Devpost to join the conversation.