Lensword

Inspiration

The objects around us are perfect vocabulary prompts, but traditional flashcards separate words from the moment we actually need them. Lensword turns a camera into a lightweight bilingual learning tool: point at a scene and learn the names of the things already in front of you.

What it does

Lensword lets a learner take or choose a photo, then identifies a small set of clearly visible, useful objects. For each result it returns a concise Simplified Chinese translation, an American English headword, IPA pronunciation, and a confidence score. Learners can hear the English pronunciation and save useful words to a private on-device word list.

How we built it

The iOS app is built with SwiftUI and AVFoundation. It resizes the selected image before upload and sends it to a Cloudflare Worker, so the OpenAI API key never ships in the app. The Worker calls OpenAI's GPT-5 nano vision model with Structured Outputs, filters low-confidence or duplicate concepts, and returns a strict bilingual vocabulary schema. A Durable Object enforces per-installation usage limits, while recognition results are cached locally for up to 30 days without storing the original photo.

Challenges

The hardest part was making the model conservative. A useful vocabulary app should not invent objects, return vague labels, or list both an object and one of its parts. We iterated on selection rules, confidence thresholds, concept deduplication, and a strict JSON schema. We also balanced image detail, latency, privacy, and API cost for a mobile workflow.

Accomplishments that we're proud of

• A complete camera-to-vocabulary flow that feels native on iPhone
• Reliable bilingual output with IPA and spoken pronunciation
• No provider credentials in the client and no server-side photo storage
• Local vocabulary persistence, result caching, and abuse-resistant quotas
• An interface ready for App Store submission

What we learned

Vision quality depends as much on restraint as recognition. Explicit exclusion rules and post-processing made the results more trustworthy than simply asking for every object in a scene. We also learned that the edge is a clean boundary for secrets, validation, and usage control.

What's next for Lensword

Next we want to add spaced repetition, example sentences grounded in the photographed scene, pronunciation practice, and more learning languages.

Built With

• swiftui