Label Lens

Snap any official letter, bill or label and get it explained in your language: what it says, what to do, and every deadline, with reminders. Built for newcomers facing bureaucracy in a new country.

Comment

Inspiration

When I moved to Italy, official letters were my biggest source of stress: tax notices, fines, residence-permit appointments, school letters. They're written in dense bureaucratic language, and missing a deadline can mean penalties. Millions of immigrants, refugees and international students across Europe face the same thing, often relying on friends or paid services just to understand one page. I wanted a tool that turns that fear into a clear to-do list, in your own language.

What it does

Take a photo of any official letter, bill, form or medicine leaflet, and Label Lens explains it in your language:

  • Plain-language summary: what it is, who sent it, what it means
  • Deadlines with date boxes and one-tap reminders (3 days and 1 day before)
  • Step-by-step "what to do": where to go, what to bring, how to pay
  • Key details: amounts, payment codes and reference numbers, copied exactly
  • Warnings and a glossary of the bureaucratic terms
  • Follow-up questions about that specific letter
  • 12 languages (English, Persian, Arabic, Ukrainian, Spanish, French, Bengali, Urdu, Chinese, Turkish, Romanian, simple Italian), with full right-to-left support for Persian, Arabic and Urdu

Dates are only taken from the document or worked out from a date printed on it (e.g. "30 days from notification"), and computed dates are clearly marked. Unreadable parts are flagged instead of guessed.

Monetization (RevenueCat): Free = 3 scans per week. Pro (pro entitlement) = unlimited scans, deadline reminders and follow-up questions. Yearly ($24.99, 1-week free trial) or Monthly ($3.99, 3-day free trial), sold through a RevenueCat Paywall, with Restore and Customer Center. Every scan has a real AI cost, so a small free tier plus a subscription keeps it sustainable.

How we built it

  • App: Expo / React Native (TypeScript), on-device image resizing, local notifications, local-only history
  • Payments: react-native-purchases + react-native-purchases-ui (RevenueCat Paywalls)
  • Backend: a small Cloudflare Worker proxy to Google Gemini with a strict JSON schema. The API key never ships in the app. Our proxy stores nothing; the photo is sent to Gemini only to read it.
  • Design: warm pastel UI, localized digits for Persian and Arabic

Challenges we ran into

  • Getting the model to copy payment codes and amounts exactly, and never invent dates
  • Computing relative deadlines ("within 60 days of notification") correctly from the date printed on the letter
  • Right-to-left layouts across the whole UI

Accomplishments that we're proud of

A crumpled Italian letter photographed outdoors comes back in a few seconds with the right issuer, the right deadlines and the exact payment code, in the user's own language. It's the app I wish I'd had when I arrived in Italy.

What we learned

Strict structured output plus careful prompting makes vision models reliable enough for high-stakes documents, and a simple, well-placed paywall shapes the product as much as the features do.

What's next for Label Lens

Google Play launch, PDF export, sharing a result with a family member or caseworker, and partnerships with NGOs and university international offices.

Demo letters in the video are fictional. Voiceover generated with ElevenLabs.

Built With

Share this project:

Updates

Submission history