ServiceSense AI

AI-powered service advisor that identifies problems, recommends the right service, estimates costs, and matches customers with suitable workers.

Comment

💡 Inspiration

In Bangladesh, households, vehicle owners, and small businesses face significant challenges whenever an appliance, motorbike, plumbing system, or electrical board breaks down:

  • Unclear Root Cause: Customers often don't know what is actually broken or which specialist they need.
  • Overcharging & Price Fraud: Informal pricing leaves customers vulnerable to unfair charges for minor repairs or spare parts.
  • Lack of Verification: Relying on unverified contacts or Facebook groups results in delays and inconsistent quality.

We built ServiceSense AI to democratize access to expert technical advice. By pairing Google Gemini's multimodal vision and natural language capabilities with local Bangladeshi market pricing and verified technician directories, we bring complete transparency, safety, and speed to every service call.

⚙️ What It Does

  1. Multimodal Problem Diagnosis: Users upload a photo of the damaged component or describe the problem in English, Bangla (বাংলা), or Banglish (or speak via microphone).
  2. Technical Assessment: Gemini AI identifies the likely technical defect, assigns an urgency level (Emergency, High, Moderate, Minor), and provides immediate safety precautions.
  3. Transparent BDT (৳) Pricing: Provides a localized price estimate range with itemized breakdown for callout fees, labor, and spare parts.
  4. Structured Work Order Scope: Auto-generates a clean checklist for the technician and pre-service questions for the homeowner.
  5. Intelligent Worker Matching: Scores and ranks verified local technicians across Dhaka, Chattogram, and Sylhet.
  6. Live Dispatch & Digital Warranty: Features interactive GPS canvas tracking, live technician chat, and downloadable 30-day warranty receipts.

🧮 Mathematical Modeling & Scoring Algorithm

🛠️ How We Built It

  • AI Engine: Integrated Google Gemini 2.0 Flash / 1.5 Flash REST API with structured JSON schema prompting (responseMimeType: "application/json").
  • Offline Resilience Engine: Built an in-browser Bangladesh Domain Expert Heuristic Engine for zero-latency fallback diagnosis.
  • Frontend Architecture: Pure HTML5, Vanilla JavaScript (ES Modules), and modern CSS3 design tokens with dark/light glassmorphism styling.
  • Speech & Audio: Browser SpeechRecognition API calibrated for bn-BD and en-US.
  • Canvas GPS Simulation: HTML5 Canvas API rendering real-time animated dispatch routes in Dhaka.

🚧 Challenges We Ran Into

  • Multilingual Banglish NLP: Handling mixed Bangla-English phrasing ("AC cholche kintu room thanda hocche na") required carefully crafted system prompts for Gemini.
  • Structured JSON Reliability: Ensuring Gemini always returned strictly formatted JSON for pricing arrays and job scope lists without markdown artifacts.
  • Zero Placeholder Polish: Building authentic vector SVG visual presets and realistic verified technician profiles across Dhaka, Chattogram, and Sylhet.

🏆 Accomplishments That We're Proud Of

  • 100% Market Calibrated: Features realistic Bangladeshi Taka (৳ BDT) rates, Dhaka neighborhoods (Dhanmondi, Gulshan, Uttara, Mirpur, Old Dhaka), and NID-verified technician credentials.
  • Instant Demo Experience: Includes 5 pre-loaded 1-click test scenarios (AC cooling failure, Refrigerator ice clog, Bathroom pipe leak, Motorbike misfire, Office DB board spark).
  • Complete End-to-End Workflow: From initial photo upload to AI analysis, worker booking, live chat, GPS tracking, and printable warranty invoice.

📚 What We Learned

  • How to structure multimodal vision prompts for Gemini 2.0 Flash to extract structured technical repairs.
  • Designing accessible, dual-language UIs tailored for South Asian emerging markets.

🔮 What's Next for ServiceSense AI

  • WhatsApp & IMO Bot Integration: Diagnose issues directly via WhatsApp voice notes.
  • bKash & Nagad Escrow Payments: Lock payment upon booking and release only when customer verifies job completion.
  • IoT Sensor Diagnostics: Connect smart energy meters to detect power spikes before breaker failure.

Built With

Share this project:

Updates

Submission history