Inspiration

One of our teammates is an international student, and deals with immigration paperwork often — RFEs, biometrics notices, status letters full of legal language that's genuinely stressful to parse even when English isn't the barrier. What kept bothering her wasn't just that these documents are confusing. It's that getting help with them almost always means handing over your name, your A-number, your case details, your address — to a person, a translation app, or an AI tool with a privacy policy you're just supposed to trust. For a population that's frequently already anxious about its legal status, asking them to expose more of themselves to get a letter explained felt like exactly the wrong tradeoff.

We wanted to build something that didn't ask people to make that tradeoff at all. Not "we promise not to misuse your data" — an actual architecture where your identity structurally cannot reach a third party in the first place, whether that third party is an AI model, our own server logs, or our own error monitoring.

What it does

Passage takes an official immigration document — an RFE, a biometrics notice, an EAD receipt, a Notice to Appear — and translates and explains it in plain language, in the reader's own language, across ten supported languages.

Before any of that happens, everything personally identifying in the document — names, A-numbers, SSNs, dates of birth, passport numbers, addresses — is detected and replaced with a placeholder token entirely inside the browser. Nothing touches the network until you explicitly press send, and what gets sent is never the original text — only the tokenized version. You can verify this yourself, live: open your browser's network tab, and the request body to our translation endpoint contains nothing but tokens like ⟦PII:NAME:1⟧.

If our automatic detection ever misses something, you're not stuck trusting it blindly — you can select the missed text yourself and redact it manually, and it's treated identically to anything caught automatically. On the way back, after Claude responds, Passage checks that every token it sent is accounted for and that no raw personal data leaked into the response. If that check fails for any reason, nothing gets displayed — not a partial result, nothing reconstructed. It fails closed.

From there, you can listen to the explanation read back to you, ask follow-up questions out loud (also redacted before they reach Claude), choose how much detail you want in the explanation, and see a list of other documents commonly relevant to your specific situation, based on what kind of document you uploaded — not a generic checklist. Throughout, Passage is explicit about one boundary it never crosses: it explains what a document is asking for. It never tells you what to write back, file, or do next. That's a deliberate legal line, not a missing feature.

How we built it

Detection runs entirely client-side: a named-entity recognition model (Xenova/bert-base-NER, via Transformers.js) running in-browser with zero network calls after the initial model download, layered with hand-written regex patterns for structured identifiers like A-numbers, SSNs, and dates. Detected spans get tokenized in the browser before anything is transmitted.

The backend is a thin proxy — it holds the Anthropic API key and forwards only tokenized text to Claude (Sonnet 4.6) for translation and explanation. It never receives raw personal data either, which matters: our own server logs, our own Sentry error events, and our own observability traces can't leak what was never in that part of the system to begin with. Redis (via Upstash) stores only an ephemeral session marker, not PII, with no persistence. Two optional Redis-backed services — Agent Memory and LangCache — handle multi-turn voice conversation context and repeated-question caching, and both are scoped to store only redacted, tokenized text, never raw values.

Sentry monitors our fail-closed validation path and our pre-send leakage scan, scrubbing common PII patterns from its own events as a second line of defense. Arize and Phoenix (dual-export, switchable per launch) track detection recall per document type with custom spans, so we have a real, measured accuracy number instead of a claim — including for our hardest detection case, non-Latin names. Deepgram handles voice input and read-back, transcribing questions which are then redacted client-side before ever reaching Claude, and synthesizing speech only from already-tokenized, PII-free explanation text.

We built the application itself using Claude Code and Cursor throughout, and our team worked in parallel across two separate forks before merging the strongest pieces of each into one final codebase.

Challenges we ran into

The most serious one surfaced midway through the build: we found that names were sometimes appearing in completely plain text, fully unredacted, in translated output. Tracing it down, we discovered our entire name-detection capability depended on a single point of failure — the NER model — and that model was silently failing to load on one team member's machine, due to a native binary built for a newer macOS version than the one actually running it. The app fell back to regex-only detection with no warning shown anywhere in the interface. Since our regex patterns had never covered names in the first place — only structured identifiers like A-numbers and dates — every name was leaking, invisibly, the entire time someone tested on that machine.

We treated this as the architectural failure it was, not a one-off bug. The fix was a real, independent regex layer for names — label-anchored patterns that work without depending on any model loading successfully — so name detection no longer has a single point of failure. We also found and fixed a quieter bug in the same investigation: a length-based filter meant to apply only to NER-sourced spans was silently dropping legitimate regex-detected name matches too, which had been suppressing detection on certain formats (ALL CAPS names, hyphenated names) independent of the NER issue entirely.

Separately, while testing translated output across languages, we caught our own system drifting past a line we'd explicitly drawn. A Spanish translation of a document with a response deadline included a line recommending the reader consult an immigration lawyer — a soft form of advice-giving, well-intentioned, but exactly the kind of thing our system prompt was supposed to prevent. We tightened the prompt to explicitly separate "stating that a deadline or consequence exists" from "recommending any course of action, including seeking legal help," and re-verified across our full test set in both languages we'd built that rule for.

Accomplishments that we're proud of

We didn't just build a privacy claim — we built one a judge, or anyone, can verify themselves in about ten seconds with their browser's devtools open, with nothing to take on faith. We're proud that when we found real failures during the build — the silent name-detection gap, the address leakage edge case, the advice-language drift in Spanish — we caught every one of them ourselves, through deliberate adversarial testing, before any of them reached a demo. We kept an honest, dated log of every failure and fix rather than quietly patching and moving on, and we think that record is some of the strongest evidence of how this project was actually built.

We're also proud of the layered failure handling itself: detection that's allowed to be imperfect because we built two independent ways to catch its mistakes — a pre-send leakage scan with manual override, and post-response validation that fails closed rather than ever showing a partial or unverified result.

What we learned

That a privacy architecture is only as strong as its quietest failure mode. The scariest bug we found in this entire build wasn't loud — it produced no error, no crash, nothing in the console. It just silently leaked, the whole time, on one specific machine. That taught us to design every detection layer assuming it will eventually fail, and to build explicit, visible checks for that failure rather than trusting any single component to always work.

We also learned that a rule like "never give advice" is far easier to write into a system prompt than to actually hold under real-world phrasing and multiple languages — it took deliberately reading raw model output, not just trusting that a check passed, to catch where our own system was drifting past a boundary we thought we'd already enforced.

What's next for Passage

Closing the one gap we're upfront about today: voice input currently reaches Deepgram as raw audio before any redaction happens on our end, so we tell users to type identifying numbers rather than say them aloud. A self-hosted speech pipeline would let voice questions get the same client-side redaction guarantee that typed and pasted text already has. We'd also like to expand full Aura-2 native voice coverage so read-back doesn't fall back to an English-accented voice for languages like Chinese, Vietnamese, or Arabic. And longer term, we want to build the citation-grounded "what does this actually mean for my specific situation" flow we deliberately scoped out of this build — it's the natural next step, but it's also exactly the kind of feature that needs to be built carefully, not quickly, given how easily it could cross from explaining a document into giving advice.

Built With

Share this project:

Updates