ReClaim AI

ReClaim AI is an AI and Machine Learning powered lost-and-found platform that combines Computer Vision, semantic matching, LLMs, and Blockchain to automate item recovery.

Comment

ReClaim AI - AI-Powered Lost & Found Management System

It is an AI and Machine Learning powered lost-and-found platform that combines Computer Vision, semantic matching, LLMs, and Blockchain to automate item recovery.

💡 Inspiration

Losing a valuable item, a wallet, a family heirloom, or an essential document, is more than an inconvenience; it is a source of profound stress and financial loss. In India's crowded urban spaces and public transport, the "Recovery Gap" is staggering:

  • The Trust Deficit: Many people find items but do not know who to trust or where to surrender them safely.
  • Manual Inefficiency: Existing lost-and-found systems rely on manual catalogs that are rarely searched effectively.
  • Identification Barriers: Describing a "black bag" accurately is hard, leading to thousands of mismatched reports.

We envisioned ReClaim AI as a digital bridge, using cutting-edge AI to automate item recognition and Blockchain to create an immutable record of trust, ensuring lost items find their way home.

🎯 What it does

ReClaim AI is an intelligent platform that simplifies the entire lost-and-found lifecycle:

  • AI-Powered Recognition: Upload an image, and the system automatically identifies, categorizes, and describes the item using visual AI.
  • CCTV Intelligence: Real-time object detection using YOLOv8 monitors camera feeds to identify and timestamp unattended items automatically.
  • Smart Semantic Matching: Uses LLMs to find matches between "Lost" and "Found" reports based on semantic similarity, even if descriptors differ.
  • Blockchain Verification: Every item handover is recorded on the Ethereum Sepolia network, providing a tamper-proof receipt of the return.
  • Conversational Assistant: A real-time AI chat interface helps users file reports through natural conversation.
  • Gamified Rewards: A credits system rewards honest finders, encouraging a trustworthy community of "ReClaimers."

🤖 Core AI Features

  • AI Visual Recognition: Users can upload an image of a lost or found item, and the system uses Computer Vision models to identify the object category, color, brand clues, and distinguishing features.

  • Generative AI Descriptions: Gemini automatically generates detailed item descriptions from uploaded images, making reports richer and easier to match.

  • Semantic AI Matching: Instead of exact keyword matching, ReClaim AI uses LLM-powered semantic similarity to connect lost and found reports even when descriptions are different.

    • Example: "black laptop bag" can match with "dark office backpack"
    • Example: "silver watch" can match with "metal wristwatch"

  • YOLOv8 CCTV Detection: CCTV feeds are analyzed in real time using YOLOv8 to identify unattended objects, timestamp them, and automatically create found-item reports.

  • Conversational AI Assistant: Users interact with a chatbot powered by Gemini to file reports naturally instead of filling out long forms.

  • AI Confidence Scoring: Every potential match is assigned a confidence score based on image similarity, text similarity, and contextual metadata.

  • Blockchain Trust Layer: Once an item is returned, the handover is recorded on Ethereum Sepolia for a tamper-proof ownership trail.

🛠️ How we built it

Technology Stack

  • Frontend: Next.js 14, React, Tailwind CSS
  • Backend (Node.js): Express, TypeScript, Ethers.js

  • Backend (Python): Flask service for YOLOv8 inference and CCTV processing

  • AI & Machine Learning:

    • Machine Learning Confidence Scoring: Ranking potential matches
    • Google Gemini 1.5 Pro: Natural language understanding, semantic matching, conversational AI, and automated description generation
    • LangGraph: Stateful AI workflows
    • Clarifai: Visual similarity matching using image embeddings
    • YOLOv8: Real-time object detection and unattended item monitoring
    • Semantic Similarity Matching: Connecting lost and found reports with different descriptions
    • Computer Vision: CCTV feed analysis and object tracking

  • Storage & Hosting:

    • Firebase Firestore: Real-time database operations
    • Cloudinary: Optimized image storage and delivery
    • Vercel: Frontend deployment
    • Render: Backend and AI service hosting

  • Blockchain & Trust Layer:

    • Ethereum Sepolia: Tamper-proof handover records
    • Solidity: Smart contracts for ownership verification
    • Ethers.js: Blockchain interactions

🔄 AI Workflow

  1. A user uploads an image or enters a text description.
  2. Gemini generates a rich description of the item.
  3. Clarifai compares image embeddings with previously uploaded items.
  4. Gemini performs semantic similarity matching across lost and found reports.
  5. YOLOv8 continuously scans CCTV footage for unattended objects.
  6. The platform ranks the most likely matches using AI confidence scores.
  7. Once verified, Blockchain records the successful handover.

Architecture

  1. Detection Layer: YOLOv8 extracts frames and identifies object classes from CCTV feeds or uploaded images.
  2. Intelligence Layer: Gemini 1.5 Pro analyzes descriptions and generates confidence scores for potential matches.
  3. Trust Layer: Smart contracts on Ethereum verify and record successful item returns.

🚧 Challenges we ran into

  1. Real-Time Synchronization: Connecting a Python-based YOLO service with a Node.js backend required a high-performance API bridge to avoid latency in CCTV detection.

  2. The Descriptive Gap: Different people describe the same item differently.

    • Example: "blue purse" vs. "navy handbag"
    • Solution: We implemented semantic embeddings using Gemini so the platform understands that similar descriptions can refer to the same object.

  3. Visual Noise in CCTV: Shadows, movement, and lighting changes created false positives.

    • Solution: We implemented keyframe extraction and confidence thresholds so only persistent objects are flagged as found items.

🏆 Accomplishments that we're proud of

  • Successfully integrated Blockchain, AI, and Computer Vision into a unified dashboard.
  • Achieved over 90% accuracy in item matching using a multi-model approach combining visual and semantic intelligence.
  • Built a seamless user experience where blockchain transactions happen in the background without confusing the user.
  • Developed a production-ready MVP within a 48-hour hackathon sprint.

📚 What we learned

  • AI Ethics: Protecting user privacy is critical when handling photos of lost items, so we implemented strict metadata stripping.
  • Trust Design: Blockchain is not just for finance, it can also strengthen civic trust and accountability.
  • Mobile-First Thinking: Most users report lost items while on the move, making mobile optimization essential.

🚀 What's next for ReClaim AI

Short-Term Goals (Next 3 Months)

  • Add regional language support including Hindi, Arabic, and other dialects.
  • Launch a Progressive Web App (PWA) with push notifications for instant "Match Found" alerts.

Medium-Term Goals (6–12 Months)

  • Integrate with local police departments and transport lost-property APIs.
  • Use Gemini multimodal capabilities to verify ownership through security questions about an item's contents.

Long-Term Vision

  • Build a global decentralized lost-and-found network where airports, malls, train stations, and public spaces can sync to a shared recovery database.

🌟 Impact Vision

Our ultimate goal is to foster a community of honesty, trust, and efficiency. By removing the friction of reporting and the uncertainty of proof, ReClaim AI is not just helping people recover lost items, it is helping reclaim a sense of safety and trust in shared public spaces.

Built With

Share this project:

Updates