Paper Bridge

upload to decode paper
result from the decode with vietnamese language

Inspiration

Every week, a kid comes home with a backpack full of paper: a permission slip, a PTA newsletter, a lunch-program letter, a flyer. Buried in those twelve pages are maybe four sentences that actually matter — $12 and a signature due Friday, reply by the 30th to get a conference slot, free-lunch application due Oct 15.

For a parent who doesn't read English comfortably, the current answer is to translate the whole stack. That gives them twelve pages in Vietnamese instead of twelve pages in English. They still don't know what to do. Missed deadlines mean a kid who stays behind on field-trip day, a lunch balance that goes negative, a conference slot that goes to someone else.

We wanted a tool that does what a bilingual older sibling does at the kitchen table: skim the pile, circle the four things that matter, and say "sign this, pay this, Friday."

What it does

Paper Bridge turns a stack of school paperwork into a short to-do list in the parent's language.

Photograph up to 8 pages — flyers, forms, newsletters. Blurry is fine.
Extract only the action items: deadlines, money, signatures, replies. Everything else is dropped.
Translate into English, Español, Tiếng Việt, or 中文 — and every item shows the exact English sentence it came from, so a bilingual relative or the teacher can check it in one glance.
Add to calendar — one tap downloads an .ics with every date; payments and signatures get a reminder two days early.
Draft the reply — for anything that needs an answer (RSVP, conference sign-up), a polite English email is already written. Edit, copy, send.

Results live at a private share link for 30 days. Photos are never stored. No account is needed; optional Google sign-in keeps a history of past stacks.

How we built it

Frontend: Next.js 16 (App Router), TypeScript, Tailwind v4, shadcn/ui, next-intl for the four locales. Deployed on Vercel.
Backend: FastAPI + Pydantic v2 on Railway. One endpoint, POST /decode, takes the images and returns a typed DecodeResult.
Vision + extraction: Gemini with structured output — the response schema is our Pydantic model, so the model can't return a shape the UI doesn't understand. One call per page in parallel, then a merge pass that de-duplicates items across pages and drafts the replies.
Deterministic where it counts: urgency (overdue / this week / later) is computed in code from the due date, never asked of the model. The client recomputes it against today, so a share link opened a week later regroups correctly.
Storage: Cloudflare KV holds only the result JSON, 30-day TTL. Images are processed in memory and discarded.
Auth (optional): Firebase Google sign-in + Firestore, frontend-only — the API never sees a user.
Process: we wrote a design spec and a task-by-task implementation plan first, then split the work: one of us on the backend, one on the frontend, meeting at the API contract. Every task was a branch + PR. We designed the screens in Pencil before writing UI code, and used AI pair-programming heavily for boilerplate, tests, and the sample-document set.

Challenges we ran into

Gemini's free tier is 20 requests/day, and linking Cloud Billing doesn't lift it — the Gemini API has its own tier gate that needs a separate upgrade in AI Studio. We lost real time to "503 high demand" errors that were actually quota.
Reply drafts attached to the wrong items. The model returns drafts by index into its item list; we sorted items by due date before mapping index → id. Every draft landed on the wrong card. Fixed by mapping first, sorting after — and a regression test that fails on the old code.
Chrome auto-translate crashed React. Chrome wraps text nodes in <font> tags; when our "Decode" button swapped its icon, React threw insertBefore … not a child of this node. Fix: translate="no" on <html> and text inside <span>.
Tiny config bites: a trailing slash in ALLOWED_ORIGIN silently broke CORS ("Failed to fetch" in the browser while the server logged 200); a trailing slash in the API URL produced //decode → 404. Both are now tolerated in code.
No real school documents to test with. We generated seven realistic ones (permission slip, PTA newsletter, lunch letter, sports flyer, plus Spanish, Vietnamese, and Chinese notices) as HTML rendered by headless Chrome, with a table of what each should extract.

Accomplishments that we're proud of

It works end to end in production: photo → four correct Vietnamese items with the right dates and amounts → .ics → drafted reply, in about 16 seconds.
Every item is auditable. Showing the source sentence was a design decision, not a feature — it's what makes a parent able to trust the output without trusting a black box.
The scope held. No accounts required, no image storage, no push notifications, no Google Calendar OAuth. We wrote down what we would not build on hour one and it kept us shipping.
A real, localized product surface: four languages across every screen including the About page, desktop and phone layouts, an account menu, rate limiting, and 22 backend tests.

What we learned

Structured output changes how you design with LLMs. When the schema is the contract, the model becomes a component with a type signature — you can test around it, fake it, and reason about failure modes.
Compute what you can compute. Anything deterministic (urgency, sorting, ids) belongs in code. The model's job is reading and translating, nothing more.
Index-based joins across a pipeline are fragile. Any reorder between producer and consumer silently corrupts them. Map early, sort late.
A spec + plan up front let two people work in parallel without stepping on each other — the API contract was the only meeting point, and integration took an hour instead of a day.
The gap between "works on localhost" and "works in production" is mostly config with trailing slashes.

What's next for Paper Bridge

Phone-camera reality: our test set is clean renders; next is skewed, shadowed, crumpled photos and a de-skew step before Gemini.
Streaming progress: the decoding screen is currently elapsed-time driven. Streaming per-page events from the backend would make it honest.
Send, not just copy: open the drafted reply directly in Gmail / the mail app, pre-addressed to the teacher when the email is on the page.
More languages: adding one is a messages/<code>.json file plus one line in the language map — Tagalog, Arabic, and Haitian Creole are the obvious next three for US districts.
Household sharing: one stack, two parents, same calendar.
Schools as the sender: a district could run every newsletter through Paper Bridge before it goes out, and attach the four-language action list to the PDF itself.