Inspiration
What it does
How we built it
Challenges we ran into
Inspiration
We all save things we mean to revisit: a concert flyer, an internship opportunity, a useful post, or an idea captured between classes. Too often, those screenshots disappear into our camera rolls instead of becoming something we act on.
AI assistants promise to help, but that convenience often starts with uploading personal content. We wanted a different approach: an assistant that understands information on your device and shares only the public details needed to find useful answers.
That became PrismLocal.
What it does
PrismLocal turns screenshots, spoken requests, and typed thoughts into practical next steps.
Upload a screenshot to extract useful details or visible links. Speak a request, review the transcript, and choose whether to search or save it. Results appear as action cards with source links, useful actions, and downloadable calendar reminders when event details are available.
A searchable local library keeps saved information accessible, while the Muse feed surfaces connections and related discoveries.
Privacy is visible throughout the experience:
- Local processing: screenshots and voice transcription are handled on-device.
- Limited web searches: Tavily receives short queries built from scrubbed public details.
- Source citations: web results link back to their original pages.
- Privacy receipts: users can inspect what left the device and what was blocked.
- Air-Gap Mode: external application searches can be disabled while local features remain available.
How we built it
We built the interface with React, TypeScript, and Vite, backed by a Python FastAPI application.
A local Qwen vision model running through Ollama interprets screenshots. A deterministic scrubber using Presidio and spaCy checks the extracted information before a query builder creates public search queries.
Tavily supplies current web results, with Exa supporting related discovery. External search requests pass through a central egress guard rather than being scattered throughout the application.
We use SQLite for local history and durable saved items. For screenshot link lists, we added local URL extraction and an optional macOS OCR pass. Incomplete or unclear URLs require review instead of silently becoming guessed links.
Challenges we faced
The hardest problem was preserving usefulness without letting private content slip into a search request. We had to distinguish public entities from personal details and enforce that boundary in code, not just in a model prompt.
Screenshots also contain imperfect information. URLs can be truncated, characters can look alike, and event times can be missing. We learned to expose uncertainty rather than present every extraction as correct.
Another challenge was deletion. A late background job or an offline browser tab could restore information after a reset. We introduced transaction checks and persistent version markers to prevent stale writes from bringing deleted data back.
What we learned
Privacy is an architectural decision. It shapes model processing, network access, storage, deletion, and the interface.
We also learned that trustworthy AI needs understandable evidence: source links, honest error states, and a clear explanation of what happened locally versus online.
What’s next
We want to improve multi-event extraction, date and time accuracy, and URL recognition across platforms. We also plan to add freshness-aware result caching so repeated requests can reuse recent public results without presenting stale information as live.
PrismLocal: You save it. We help you make it happen.
