Inspiration

Bills, leases, tax notices and insurance denials are the documents people most need help understanding, and the ones they should least paste into a cloud chatbot. They are full of names, addresses, ID numbers and account numbers. On-device models are private but small; cloud models are strong but see everything. Blindfold uses both, each for the part it is right for.

What it does

  1. Scan on device. Gemini Nano in Chrome, plus deterministic rules for values with a fixed shape (phone numbers, card numbers with a Luhn check, emails, SSNs), finds private values. A second Nano pass audits the already-redacted text as a safety net before anything is sent.
  2. You review. The cloud's view is shown beside the original, with placeholders like [PERSON_1], [ID_1], [ADDRESS_1]. Click any value to let it through or keep it hidden; select any text to hide more. The Ask button is disabled while a known private value is visible.
  3. Reason in the cloud. Only the redacted text goes to Gemini. Amounts, dates and organisation names stay, because the model needs them to think.
  4. Restore on device. The answer comes back with placeholders and the real values are put back locally. A wire monitor shows the exact request body the SDK sent to Google (in the demo: 1,574 bytes, 0 private values).

It also reads a photo of a document on-device (Gemini Nano multimodal input), so the image never leaves the tab either.

How we use Firebase AI Logic (hybrid inference)

All inference goes through firebase/ai (src/ai.js). Each step uses the hybrid mode that matches its risk:

Step Mode Why
Detect and audit private values, read a document photo ONLY_ON_DEVICE Raw text must never leave the device, so these steps can never fall back to the cloud. The detect and audit steps also throw if inferenceSource is not on_device.
Deep analysis ONLY_IN_CLOUD Needs the strongest model; only ever sees redacted text.
Quick answer PREFER_ON_DEVICE Safe on either side because its input is already redacted.

Other Firebase pieces: App Check with reCAPTCHA Enterprise on the hosted build (debug token only on localhost, never in the hosted bundle), Firebase Hosting, and the free Spark plan. If Gemini Nano is unavailable, Blindfold falls back to the rule-based scanner and says so on screen.

Results

30 synthetic documents (medical bills, leases, tax notices, claim denials, letters) with 180 private values and 120 values that must survive. Everything runs on-device with Gemini Nano.

Redaction Private values leaked Documents fully clean
none (raw text to the cloud) 180 / 180 (100%) 0 / 30
rules only 30 (16.7%) 18 / 30
Gemini Nano only 18 (10.0%) 12 / 30
rules + Gemini Nano 4 (2.2%) 26 / 30

Harness and raw results are in the repo (bench/). Before filtering model-reported amounts and plain dates, the same pipeline lost 42% of the useful values; after the filter it loses 1.7% (rules + Nano).

How we built it

Vanilla JavaScript and Vite, Firebase JS SDK 13 (firebase/ai, firebase/app-check). Pure functions for detection, redaction, restore and leak checking are covered by unit tests (npm test), and an end-to-end run against the hosted app (Chrome with Gemini Nano) checks the full flow.

What we built during the submission period

Everything. The repository's first commit is from 2026-11-12, after the Submission Period started. No pre-existing code was used; the only third-party code is the open-source libraries listed in package.json. The sample documents and benchmark data are synthetic, with invented people, addresses and numbers.

What's next

Chrome extension form so the scan runs on any page; more document types; image-region redaction for photos.

Testing instructions

No login or payment needed. Open https://blindfold-hybrid.web.app in desktop Chrome (the full on-device flow needs Gemini Nano; if the status pill says "download needed", click "Enable on-device AI" once). Pick a sample document, press Scan on device, then Ask. Without Gemini Nano the app still works with the rule-based scanner and shows a banner explaining the lower protection level. To run locally: npm install, copy .env.example to .env.local with your own Firebase web app config, npm run dev, npm test.

Built With

  • chrome
  • firebase-ai-logic
  • firebase-app-check
  • firebase-hosting
  • gemini-nano
  • javascript
  • vite
Share this project:

Updates

Submission history