shrammitra-ai

ShramMitra AI is a multilingual WhatsApp assistant using RAG + Amazon Bedrock (Claude 3 Sonnet) to answer migrant workers’ labour law queries in 7 Indian languages.

Comment

ShramMitra AI: Comprehensive Project Overview

This document serves as a deep dive into ShramMitra AI to help developers and stakeholders fully understand the inner workings, data flows, and architectural decisions behind the system.

🎯 1. Core Mission & Objective

ShramMitra AI is a localized, multilingual WhatsApp-based AI assistant. It is designed to bridge the information gap for migrant workers in Bengaluru, India. Official labour rights, minimum wage laws, and complaint procedures are often buried in complex English PDFs. ShramMitra AI uses RAG (Retrieval-Augmented Generation) to extract this data from official sources and serve it to workers in 7 native languages (Hindi, Kannada, Tamil, Telugu, Bengali, Odia, English) directly via WhatsApp.

🏗️ 2. Architectural Deep Dive

Here is the high-level system architecture, illustrating the flow from a worker's WhatsApp message to the internal RAG and LLM components, and finally the administrative dashboard:

flowchart TD
    subgraph External
        W[Worker on WhatsApp]
    end

    subgraph "WhatsApp Cloud"
        WA[Meta WhatsApp Business API]
    end

    subgraph "ShramMitra Backend (FastAPI)"
        API[Webhook API Gateway]
        Auth[Rate Limiting & Auth]
        Svc[Services: Lang Detect / Translation]
        RAG[RAG Orchestrator]
    end

    subgraph "AWS AI Services"
        Voice[AWS Transcribe & Polly]
        LLM[Amazon Bedrock: Claude 3]
    end

    subgraph "Storage & Databases"
        Redis[(Redis: Session Cache)]
        PG[(PostgreSQL: Chat Logs)]
        ES[(Elasticsearch: Vector DB)]
    end

    subgraph "Data Ingestion Pipeline"
        Crawler[Elastic Open Crawler]
        PDF[PDF Parser & Chunker]
        Jina[Jina v5 Embeddings]
    end

    subgraph "Admin System"
        NextJS[Next.js Admin Dashboard]
        Admin[NGO / System Admin]
    end

    %% Flow: User to Backend
    W <-->|Text / Voice Notes| WA
    WA <-->|HTTPS Webhooks| API

    %% Backend internal
    API --> Auth
    Auth --> Svc
    Svc <-->|Convert Audio| Voice
    Svc --> RAG

    %% Backend to Data
    RAG <-->|Fetch Context| Redis
    RAG -->|Log Data| PG
    RAG <-->|Semantic Search| ES
    RAG <-->|Prompt + Context| LLM

    %% Ingestion Flow
    Crawler --> PDF
    PDF --> Jina
    Jina -->|Index Chunks| ES

    %% Admin Flow
    Admin --> NextJS
    NextJS <-->|Fetch Analytics| API

The project is structured into three highly decoupled systems:

A. The Backend Engine (FastAPI)

  • Role: The brain of the operation. It receives WhatsApp webhooks, orchestrates the AI agents, and formats responses.
  • Technologies: Python 3.11, FastAPI, SQLAlchemy, Redis.
  • Key Responsibilities:
    • Session Management (Redis): Remembers the context of a conversation for up to 1 hour so workers can have an ongoing chat.
    • Authentication & Rate Limiting: Prevents spam (e.g., limits a phone number to 30 requests per minute).
    • Audio Processing: Integrates with AWS Transcribe to convert voice notes to text, and AWS Polly to send audio replies back to illiterate workers.

B. The AI & Search Layer (Bedrock + Elasticsearch)

  • Role: Understanding the query, searching the laws, and generating the answer.
  • Technologies: Amazon Bedrock (Claude 3 Sonnet), Elasticsearch 9.x, Jina v5 Embeddings.
  • How it works:
    • The backend takes the worker's translated text and generates a semantic vector using Jina v5.
    • It searches Elasticsearch for the top 5 most relevant chunks of labour law text.
    • It sends the retrieved context + the worker's question to Claude 3 Sonnet via AWS Bedrock to generate a simple, grounded, and empathetic response.

C. The Data Ingestion Pipeline

  • Role: Keeps the AI's knowledge base updated with verified facts.
  • Technologies: Python (pdfplumber, pypdf), Elastic Open Crawler.
  • How it works: A scheduled job crawls the Karnataka Labour Department website, downloads PDFs, splits them into paragraphs (chunking), generates embeddings, and pushes them into Elasticsearch.

D. The Admin Dashboard (Frontend)

  • Role: Allows administrators and NGOs to monitor the system.
  • Technologies: Next.js 14, TailwindCSS, shadcn/ui.
  • Features: View conversation logs, track analytics (most used languages, common topics), and monitor the health of the crawler.

🔄 3. End-to-End Workflow (Step-by-Step)

Here is exactly what happens when a worker sends a message:

  1. Incoming Message: A worker sends "Mera malik PF nahi de raha" (My boss isn't paying PF) on WhatsApp.
  2. Webhook Trigger: WhatsApp hits the FastAPI /webhook/whatsapp endpoint.
  3. Language Detection: The system detects the language as Hindi.
  4. Translation (Optional): If needed, the query is internally translated to English for better search accuracy.
  5. Vector Search: The query is converted to an embedding. Elasticsearch finds the official rules for EPFO (Employees' Provident Fund Organisation).
  6. Prompt Assembly: FastAPI combines the retrieved EPFO rules and the worker's original Hindi message into a strict prompt.
  7. LLM Generation: Claude 3 reads the prompt and generates a response in Hindi, strictly based only on the retrieved text.
  8. Outgoing Message: FastAPI sends the generated Hindi response back to the worker via the WhatsApp API.

💬 4. Sample Input & Output

Scenario 1: Minimum Wage Inquiry (Hindi)

Input (Worker on WhatsApp):

"मैं एक निर्माण मजदूर हूँ। एक दिन की न्यूनतम मजदूरी कितनी है?" (Translation: I am a construction worker. What is the minimum wage for a day?)

System Action:

  • Detects Hindi.
  • Retrieves "Minimum Wages Act - Karnataka Construction Workers Notification".

Output (ShramMitra AI):

"नमस्ते। कर्नाटक श्रम विभाग के अनुसार, एक अकुशल निर्माण मजदूर के लिए न्यूनतम मजदूरी ₹436 प्रति दिन (प्लस VDA) है।

यदि आपको इससे कम मिल रहा है, तो आप 1800-425-1200 (निःशुल्क) पर कॉल करके शिकायत दर्ज कर सकते हैं। क्या आप जानना चाहते हैं कि शिकायत फॉर्म कैसे भरें?"

Scenario 2: Withheld Pay (Audio Message in Kannada)

Input (Worker sends a Voice Note):

[Audio Note: "ನನ್ನ ಮಾಲೀಕರು ಎರಡು ತಿಂಗಳಿಂದ ಸಂಬಳ ಕೊಟ್ಟಿಲ್ಲ."] (Translation: My employer hasn't paid salary for two months.)

System Action:

  • AWS Transcribe converts Kannada Audio to Text.
  • Retrieves "Payment of Wages Act".

Output (ShramMitra AI sends text + audio):

"ನಿಮ್ಮ ಸಂಬಳವನ್ನು ತಡೆಹಿಡಿಯುವುದು ಕಾನೂನುಬಾಹಿರ. ಪಾವತಿ ಕಾಯ್ದೆ 1936 ರ ಅಡಿಯಲ್ಲಿ, ನೀವು ಕಾರ್ಮಿಕ ಅಧಿಕಾರಿಗೆ ದೂರು ನೀಡಬಹುದು. ದಯವಿಟ್ಟು ನಿಮ್ಮ ಹತ್ತಿರದ ಕಾರ್ಮಿಕ ಕಚೇರಿಯನ್ನು ಸಂಪರ್ಕಿಸಿ." [Attached: Kannada Audio version of the text via AWS Polly]

🛡️ 5. Safety & Guardrails

Because this system deals with legal rights, AI hallucinations must be strictly avoided. The architecture implements several guardrails:

  1. Strict RAG: Claude 3 is instructed to explicitly say "I don't know" if the answer isn't in the Elasticsearch database.
  2. No External Browsing: The agent cannot search the live internet; it only reads official, pre-vetted PDFs.
  3. Disclaimer Injection: Every new conversation starts with a legal disclaimer stating the bot is an informational tool, not a human lawyer.
  4. PII Masking: Names and addresses detected in the chat are never logged to the database.

🚀 6. Use Cases & Target Audience

  • Construction & Gig Workers: Checking minimum wage rates and accident compensation procedures.
  • Domestic Helpers: Understanding maternity benefits under ESIC.
  • NGOs & Activists: Using the admin dashboard to identify trends in labour violations across different areas of Bengaluru.

Built With

  • 3
  • access
  • accurate
  • actions:
  • ai
  • amazon-web-services
  • and
  • and-webhook-management.-aws-cloudwatch-&-secrets-manager:-monitoring
  • api-gateway:-core-hosting
  • apis
  • automatically
  • automation.
  • bedrock
  • business
  • ci/cd
  • claude
  • communication
  • compose:
  • containerization
  • crawler:
  • credentials.
  • deployment.
  • development
  • devops
  • docker
  • elastic
  • embeddings.
  • embeddings:
  • file-storage
  • for
  • generating
  • github
  • government
  • haiku
  • highly
  • infrastructure-as-code
  • interface
  • jina
  • kibana
  • language
  • large
  • legal
  • local
  • logging
  • main
  • management.
  • meta
  • model
  • models:
  • multilingual
  • official
  • open
  • pipeline
  • power
  • provision
  • python
  • resources.
  • responses
  • s3
  • scraping
  • securing
  • security
  • sites
  • sonnet
  • terraform:
  • text
  • the
  • to
  • translations.
  • used
  • v5
  • whatsapp
  • workers.
Share this project:

Updates