SOVEREIGN — Private Knowledge Vault with On-Device AI

Turn screenshots and documents into searchable, sourced facts with Gemini Nano running in Chrome. Your data stays on your device; cloud AI runs only when you approve it.

Comment

Inspiration

Screenshots, PDFs and notes pile up full of facts we need later — but sending them to a cloud AI means handing over personal data. We wanted an assistant that reads your documents on your own machine and only touches the cloud when you explicitly say yes.

What it does

SOVEREIGN is a private knowledge vault in Chrome. You drop in screenshots, PDFs or Word files; it runs OCR locally, then uses Gemini Nano (Chrome built-in AI) through Firebase AI Logic to extract structured facts. Every fact carries an exact quote from its source, so you can verify it. If on-device AI isn't available, the app says so honestly and offers cloud Gemini — but only after you approve it for that item, and the result is labeled accordingly.

How we built it

  • Vite + React + TypeScript, Firebase JS SDK (@firebase/ai) with InferenceMode.ONLY_ON_DEVICE and an approval-gated ONLY_IN_CLOUD path
  • JSON-schema responseConstraint for structured extraction
  • Tesseract.js (self-hosted WASM + language data) for offline OCR; pdf.js and mammoth for documents
  • IndexedDB vault encrypted with AES-GCM via WebCrypto (non-extractable key)
  • Shared JSON-schema contracts and fixtures, with canonical JSON and integrity checks for provenance

Challenges

We started on Android with Gemini Nano, but our emulator had no AICore, our Pixel 7 Pro isn't on the supported device list, and a streamed Pixel 10 Pro returned a binding failure. Chrome's built-in model on an M1 Mac worked first try, so we moved to the web. On-device inference is real but slow (~15 s to create a session and ~15 s per prompt), with a 9,216-token input limit — so we had to design chunking and progress UX around it.

What we learned

Hybrid AI is a product decision, not a toggle: the on-device path is what makes the privacy promise true, and the cloud path has to be visibly opt-in to keep it.

What's next

PWA share target, the paused Android client once supported hardware is available, and richer search over the vault.

Built With

Share this project:

Updates

Submission history