ChequeAI

Analyze and extract cheque details instantly—100% on-device, no cloud, no compromise on privacy.

Comment

💡 Inspiration

In financial institutions, cheque verification and processing still depend on manual effort or costly legacy OCR systems. Even modern cloud-based AI tools introduce data privacy risks, as sensitive financial information leaves the organization's secure environment.

We were inspired by the potential of Chrome’s Built-in AI APIs — especially Gemini Nano — to move AI computation on-device. This lets banks and financial teams analyze cheques instantly, securely, and cost-free — all within the browser, no data ever leaves the device.

⚙️ What It Does

ChequeAI is a fully client-side web application that:

  • Analyzes cheque images locally using Chrome’s Built-in AI (Gemini Nano via LanguageModel.prompt()).
  • Extracts structured data—including Payee, Amount (words & figures), IFSC, Date, and Cheque Number etc—using on-device multimodal - Prompt API Gemini Nano + Zod Schema (no cloud OCR).
  • Smart Review Interface: Editable form synced with image preview (zoom/rotate/mirror) so users can verify or correct extracted fields.
  • Validates data integrity by detecting inconsistencies like amount mismatches or missing fields.
  • Supports voice and text remarks that feed into the AI analysis.
  • Generates a forensic AI report (via Prompt API Multimodel)
  • Generates Summary via Summarizer API
  • Translates the Summary on-device into multiple languages (via Translator.translate()).
  • Runs 100% offline—no data leaves the browser, aligning with Zero-Trust and Privacy-by-Design principles.

🛠️ How We Built It

  • Frontend Framework: Next.js 15+ (App Router) + TypeScript
  • Styling: Tailwind CSS v4 + shadcn/ui
  • AI Engine: Chrome Built-in AI (LanguageModel - Prompt API, LanguageDetector API, Translator API, Summarizer API)

  • Prompt APIs Used:

    • prompt() → field extraction (OCR + responseConstraint Schema) & analysis
    • summarize() → generate summarize of final reports
    • translate() → instant translation of summary

Architecture Overview

[ User Uploads Cheque ]
        ↓
[ Next.js Frontend (Client Only) ]
        ↓
[ Chrome Built-in AI (Gemini Nano) ]
        ↓
[ Structured Fields]
       ↓
[Editable Form & Preview]  
       ↓
[Remarks - Text & Audio Input (Optional)]
       ↓
[Analysis - AI Report → Summary (on Demand)  → Translation (On Demand) ]
  • Zero Backend: No server or cloud components.
  • No API Keys: Uses only built-in browser AI models.
  • All Processing: Performed securely within Chrome’s sandbox.

🚧 Challenges We Ran Into

  • Managing varied cheque formats and handwriting styles using prompt-based extraction — Gemini Nano often produced inconsistent data, leading to integrity issues during extraction, summarization, and translation (both requiring user interaction, limiting automation).
  • Dealing with the large model size, which increases first-time load time.
  • Ensuring a fully offline experience while keeping the UI fast, responsive, and modern.
  • Designing a privacy-first architecture — no cloud, no telemetry, no cookies.
  • Integrating multiple Built-in AI APIs (Prompt API - LanguageModel, Summarizer Translator) seamlessly within the browser environment.
  • Debugging and testing with experimental Chrome flags for Gemini Nano support.
  • Handling undocumented TypeScript types, requiring manual exploration and experimentation.
  • On Summary Markdown in Indian Language not generate correct format

🏅 Accomplishments We’re Proud Of

  • First working prototype of a fully offline financial document analyzer.
  • Achieved complete on-device AI pipeline — no network calls at all.
  • Demonstrated real-time field extraction accuracy > 85% for standard cheque formats.
  • Built a modern UI with smooth step-based workflow (Upload → Review → Remarks → Analyze → Summary → Translate in Local Language).
  • Proved enterprise-ready compliance through Zero-Trust architecture.
  • Successfully integrated experimental Chrome AI APIs.

📚 What We Learned

  • On-device AI is ready for enterprise-grade privacy use cases.
  • Building with Built-in AI (Gemini Nano, Prompt API, Translator) is powerful but still evolving — documentation gaps require deep experimentation.
  • Prompt design and schema validation with structured-outputs are key to maintaining data accuracy in multimodal extraction.
  • Offline-first web apps can be powerful and secure when paired with browser-native AI.
  • Chrome’s Built-in AI APIs offer a new paradigm — lightweight AI without infrastructure.
  • Strong documentation and developer ergonomics make adoption easier for teams.

🚀 What’s Next for ChequeAI

  • Enhanced On-Device AI Models – Optimize Gemini Nano pipelines for faster, smaller, and more accurate cheque data extraction.
  • Add signature & handwriting verification modules.
  • Enhance forensic analysis with MICR line validation, signature presence detection, and tamper-heuristic rules.
  • Support batch upload and analysis dashboard for auditors.
  • Collaborate with Chrome AI & fintech partners to promote secure, privacy-first AI adoption.

Built With

Share this project:

Updates