Snap Sort

Mobile users waste time scrolling through endless screenshots. SnapSort uses AI vision to automatically organize, extract text from, and index your images. No manual tagging or folders just search.

Comment

SnapSort

Inspiration

Most camera rolls tell the same story: hundreds of screenshots saved with the intention of finding them later. A recipe gets buried under memes, a product link disappears among receipts, and code snippets end up mixed with social media posts.

The native Photos app treats screenshots as ordinary images. There is no meaningful categorization, metadata, or search capability.

The release of Gemini 2.5 Flash-Lite with vision capabilities created an opportunity to solve this problem. The model is fast and affordable enough to process large numbers of screenshots while understanding their contents. Instead of simply describing an image, it can classify it, extract text, generate tags, and make screenshots searchable.

What It Does

SnapSort automatically converts screenshots into a searchable, organized library.

Features

  • Select one or many screenshots from your photo library.
  • Analyze screenshots using Gemini 2.5 Flash-Lite.

  • Generate structured metadata for each image, including:

    • Category
    • Title
    • Tags
    • Extracted text
    • Summary

  • Browse screenshots in a responsive dark-themed grid.

  • Filter by 11 categories:

    • Tweet
    • Article
    • Recipe
    • Product
    • Code
    • Receipt
    • Meme
    • Chat
    • Outfit
    • Shoes
    • Other

  • Search across:

    • Titles
    • Tags
    • Extracted text
    • Summaries

  • Open a detail view showing:

    • Original image
    • AI-generated summary
    • Extracted text with one-tap copy
    • Tag cloud

  • Retry failed analyses without selecting the image again.

  • Delete screenshots through a confirmation prompt.

How We Built It

Layer Technology
Mobile App Expo React Native (iOS, Android, Web)
Routing Expo Router (file-system based)
Navigation & UI React Native Animated API, LinearGradient, SafeAreaContext
Image Selection expo-image-picker (base64 output)
Backend Express 5 (Node.js)
AI Gemini 2.5 Flash-Lite via REST API
Persistence AsyncStorage
State Management React Context + useReducer
Build System pnpm Monorepo, TypeScript 5.9, Expo Router 6

The architecture keeps the Gemini API key on the server. The Expo application sends base64 image data to an Express proxy, which forwards requests to Gemini and returns structured JSON responses. The API key is never exposed in the client application.

Challenges We Ran Into

Payload Size

Base64 encoding increases image size by roughly 33%, causing typical screenshots to exceed Express's default 100 KB request limit.

Solution: Increased the request body limit to 20 MB.

Structured JSON Responses

Even with strict prompting, Gemini occasionally wrapped responses in Markdown code fences.

Solution: Added a post-processing step to remove formatting before parsing JSON.

Responsive Layout

Using Dimensions.get('window') at module load time produced fixed card widths that did not adapt to screen size changes.

Solution: Replaced it with useWindowDimensions(), allowing layouts to recalculate dynamically.

Retry Without Re-Selecting Images

Users should not need to reopen the photo library because of a temporary network failure.

Solution: Implemented an in-memory Map keyed by card ID to cache base64 image data during the session, enabling instant retries.

Rate Limiting

Sending multiple images simultaneously risked exceeding API limits.

Solution: Processed images sequentially with a four-second delay between requests, preserving both reliability and meaningful queue progress feedback.

Accomplishments

Automatic Classification

Users can drop in a screenshot and immediately receive a categorized, tagged, titled, and searchable entry without manual input.

Reliable Text Extraction

Recipes, receipts, code snippets, tweets, and other screenshot types are extracted and stored as searchable text.

Seamless Retry Experience

Failed analyses can be retried instantly using the session cache, eliminating the need to select images again.

Cross-Platform Support

The same Expo codebase runs on mobile and web platforms, adapting smoothly to different screen sizes.

Polished User Experience

Animations provide visual feedback throughout the workflow:

  • Shimmer effects for queued items
  • Pulse animations during analysis
  • Spring transitions for bottom sheets

These interactions make processing feel responsive, even when requests take time.

What We Learned

Vision Models Are Strong Zero-Shot Classifiers

Gemini 2.5 Flash-Lite accurately identified screenshot types such as tweets, code editors, and e-commerce product pages without fine-tuning or example-driven prompting.

Structured Prompting Is Critical

The difference between a descriptive prompt and a strictly defined JSON schema determines whether a model output is suitable for production use.

Base64 Does Not Scale Well

For larger systems, uploading images to object storage and passing URLs to the model would be more efficient than transmitting base64 payloads.

In-Memory Caching Is Underrated

A simple useRef containing a Map provided an effective way to preserve temporary image data without introducing additional dependencies or persistent storage.

What's Next

On-Device OCR

Integrate ML Kit or platform-native vision frameworks to support offline text extraction, reserving Gemini for classification tasks.

Cloud Photo Integration

Automatically process new screenshots from iCloud or Google Photos as they are created.

Smart Deduplication

Detect visually similar screenshots and consolidate duplicates.

Note-Taking Integrations

Export extracted content directly to Apple Notes, Notion, or Obsidian.

Shared Collections

Allow users to generate shareable collections such as recipe libraries, research folders, or shopping lists.

Semantic Search

Move beyond keyword matching by introducing vector embeddings, enabling searches based on meaning rather than exact text matches.

Built With

Share this project:

Updates