Zoro

Zoro: Even we don't get lost finding your stuff!

Comment

Inspiration

On construction sites and factory floors, lost tools and equipment aren't just inconvenient—they're costly and dangerous. A misplaced power drill halts work. Lost safety gear creates hazards. Workers waste valuable time digging through cluttered lost and found bins, often with dirty hands and time pressure.

Traditional systems fail in industrial environments: paper logs get damaged, spreadsheets require computer access, and exposing full inventories invites false claims for expensive equipment.

We built Zoro for the workers—a hands-free, AI-powered system where you can snap a photo of what you lost or simply describe it, and let the system find it for you. No scrolling through lists. No false claims. Just fast, secure matching.

What it does

Zoro is an intelligent lost and found assistant designed for industrial environments:

  • Worker-friendly inquiries: Describe your lost tool in plain language ("my yellow DeWalt drill with tape on the handle") or snap a quick photo—no forms to fill out
  • Smart matching engine: Combines tag-based similarity, keyword extraction, and vision AI (Google Gemini) to compare submissions against a secure inventory
  • Intelligent narrowing: When multiple matches exist, Zoro asks targeted follow-up questions ("Does it have any scratches?" "What brand?") to pinpoint the right item
  • Secure claim process: Workers verify ownership with their name, employee ID (matricule), and email before claiming
  • Status tracking: Workers can follow their inquiry: Submitted → Under Review → Matched → Resolved
  • Automated notifications: Emails confirm submissions and alert supervisors when claims are made

How we built it

Frontend:

  • Next.js 16 with React 19 and TypeScript
  • Custom "Liquid Glass" UI optimized for readability on mobile devices
  • Framer Motion for smooth animations
  • Embla Carousel for browsing matched items

Backend:

  • tRPC for type-safe API with real-time streaming
  • Drizzle ORM with PostgreSQL
  • Supabase for secure image storage
  • Better Auth for authentication

AI/ML:

  • Vercel AI SDK as LLM abstraction layer
  • OpenRouter gateway routing to:
    • GPT-4o-mini for conversational reasoning
    • Google Gemini 3.5 Flash for vision-based image comparison

Challenges we ran into

  • Distinguishing similar tools: Many workers have identical equipment (same brand, same model). Our question-generation system had to learn to ask about unique identifiers like scratches, stickers, or tape markings
  • Real-time streaming with tool calls: Coordinating tRPC subscriptions with AI tool execution while keeping the UI responsive required careful state management

Accomplishments that we're proud of

  • Photo-first UX: Snap a picture, get results. No typing required if you don't want to
  • Smart distinguishing questions: Zoro asks about unique features (tape, stickers, wear marks) that actually help identify ownership
  • Employee ID integration: Matricule-based claims create accountability and audit trails
  • Works on mobile: Designed for workers accessing from phones on the factory floor

## What we learned

  • Industrial UX is different: What works in consumer apps doesn't work for workers with gloves, limited time, and harsh lighting
  • Vision AI handles real-world photos: Gemini can match dusty, scratched tools surprisingly well with proper prompting
  • Prompt engineering matters: Small tweaks to system prompts dramatically changed how well Zoro understood tool descriptions
  • tRPC + streaming: Type-safe real-time subscriptions made our chat implementation clean and maintainable
  • Scope ruthlessly: Focusing on core matching let us polish what matters for the demo

## What's next for Zoro

  • Supervisor dashboard: Full admin portal to manage inventory, review claims, and track equipment
  • Fraud prevention: Ownership verification questions based on item details, confidence scoring, and duplicate detection
  • Tool check-out integration: Connect with existing equipment management systems
  • Multi-language support: French/English for Quebec factory workers
  • Offline mode: Cache functionality for areas with poor connectivity
  • Analytics: Track loss patterns to help sites improve equipment management and reduce losses

Built With

Share this project:

Updates