LostLens

Inspiration

Losing a personal item is frustrating. At a lost and found office, manually tracking what is lost and its description is a tedious time consuming task. Then, when someone else inevitability goes to search for it lost-and-found boards that rely on strict keyword matching fail. .A user searching for a "dark blue backpack" won't match a found report titled "Navy Herschel Bag," even if they are the exact same item.

We created LostLens to bridge this gap. By combining multimodal vector search with agentic AI reasoning, LostLens creates an intuitive two-way ecosystem that brings true semantic understanding to lost and found management.

What it does

LostLens features a streamlined interface divided into two core operational workflows:

Report Found Item (Ingestion): Anyone who finds an item simply uploads a photo and types brief notes (e.g., location or context). LostLens extracts structured metadata, processes the image and context into a multimodal vector embedding, and registers the item into the database
Find Lost Item (Search & Retrieval): Users looking for lost items describe what they lost in natural language or upload a reference image. LostLens converts the input into a search vector, queries stored items using high-speed vector similarity, and presents ranked match cards complete with percentage scores and human-readable reasoning (e.g., "✓ Same item type," "✓ Matching color," "✓ Distinct red keychain identified").

How we built it

LostLens utilizes a hybrid pipeline engineered for sub-second retrieval speed and high accuracy:

Database Layer (TiDB Cloud): We stored all records in a TiDB Serverless cluster utilizing TiDB Vector Search. Using an HNSW vector index operating over a 768-dimensional vector space (VEC_COSINE_DISTANCE), TiDB performs sub-10ms candidate retrieval across all stored reports.
Intelligence Layer (Google Gemini API): We used text-embedding-001 / Gemini multimodal embeddings to map images and text into a unified vector space, ensuring visual and semantic traits are captured together. For match evaluation, we used gemini-3.1-flash-lite as an intelligent Matcher Agent.
Backend Orchestration (Python & FastAPI): Built on Python, the backend handles image processing, payload formatting, and coordinates the flow between TiDB and Gemini.
Frontend: A clean, responsive user interface separating the Report and Find sections to reduce user friction.

Challenges we ran into

Asymmetric Data Flow: Designing the architecture so that the "Found" workflow acts strictly as an efficient data ingestion step while the "Lost" workflow performs the active similarity search required careful separation of our API functions and vector task types (RETRIEVAL_DOCUMENT vs. RETRIEVAL_QUERY).
Columnar Replica Configuration in TiDB: Enabling vector indexing on TiDB required configuring TiFlash columnar storage replicas (ADD_COLUMNAR_REPLICA_ON_DEMAND), which required updating our SQL schema execution strategy during development.
Multimodal Alignment: Ensuring that text descriptions (e.g., "lost at central library") and visual features from images were properly balanced in the embedding vector so neither modality dominated the search result.

Accomplishments that we're proud of

Sub-Second End-to-End Pipeline: We successfully combined TiDB's database retrieval (<10ms) with Gemini's agentic evaluation to deliver fully analyzed match results in under a second.
Meaningful AI Reasoning: Instead of just returning raw distance metrics, our Gemini agent provides actionable, natural language justifications explaining why an item matched.
Clean Asymmetric Design: Building a dual-section application where "Found" item uploads remain fast and lightweight, saving resource-heavy reasoning for actual search queries.

What we learned

How to combine relational data, JSON metadata, and high-dimensional vector embeddings within a single TiDB Cloud database.
The advantage of a hybrid AI architecture: utilizing database vector search for fast candidate filtering, and LLM agents for fine-grained reasoning and reranking.
Best practices for handling raw image binaries in Python and converting them into lightweight Base64 payloads and vector representations rather than cluttering database storage.

What's next for LostLens

Geo-Spatial Filtering: Integrating real-time map radius filtering in TiDB alongside vector search to narrow down matches by precise GPS coordinates.
Automated Claim & Verification System: Adding a secure dynamic Q&A prompt generated by Gemini (e.g., "What wallpaper is on the locked phone screen?") so owners can prove ownership before contacting the finder.
Push Notifications: Alerting users automatically when a newly submitted "Found" item