MSME-agent

Empowering small business stores with a Gemini-powered multilingual agent that parses chat/voice orders, automates inventory, and handles supply reorders via UiPath orchestration.

Comment

Inspiration

Kirana stores are the backbone of retail in India, processing billions of orders daily. However, store owners spend hours manually translating messy, mixed-language (Hinglish/Hindi/English) WhatsApp texts, handwritten notes, and voice orders into inventory records. We wanted to build an automated, zero-effort AI clerk that digitizes this workflow, freeing up shop owners to focus on customer service.

What it does

The MSME Order + Inventory Agent completely automates order processing:

  1. Multilingual AI Parsing: Extracts items and quantities from Hinglish/Hindi text (like "Bhai 15 kg atta and 10 ltr tel bhej do") using Google Gemini.
  2. Real-time Inventory & Shortage Logic: Evaluates orders against stock, reserving available items and generating Purchase Requests (PRs) for shortages.
  3. Approval Flow: Routes high-value reorders for review, simulating a UiPath Action Center approval queue.
  4. WhatsApp Sharing & Invoices: Generates digital receipts that owners can share directly via WhatsApp with a single click.
  5. Speech-to-Text: Features voice-input simulation using the Web Speech API configured for Hinglish/Indian accents.

How we built it

  • Backend: FastAPI (Python) running a 6-step agent pipeline integrated with SQLite.
  • Frontend: A high-end, responsive dark glassmorphic dashboard built in Vanilla HTML5, CSS3, and JS.
  • AI Orchestration: Powered by Google Gemini (gemini-2.0-flash) via the google-genai SDK, with a robust fallback regex matching engine to handle rate-limiting or network issues gracefully.
  • Enterprise Automation: Simulated endpoints designed to integrate directly with UiPath Maestro BPMN and Action Center.

Challenges we ran into

Handling colloquial Hinglish terminology and abbreviations was challenging. We solved this by providing Gemini with specific contextual system prompts mapping informal Hindi names (like cheeni, tel, chawal) to standardized catalog names. We also hit API rate limits during quick sequential testing, leading us to design a robust warning pipeline state and offline local regex parser fallback.

Accomplishments that we're proud of

  • Creating a fully functional, self-contained reorder approval screen so judges can simulate UiPath Orchestrator workflow logic instantly.
  • Seamless Hinglish voice-to-text translation directly inside the web browser.
  • Pushing stock replenishment database synchronization so that approving a reorder immediately updates stock quantities and charts.

What we learned

Building a hybrid system that pairs flexible LLMs with structured databases is powerful, but resilience fallbacks (like our regex order extractor) are absolutely critical for production-ready MSME tools.

What's next for MSME agent

We plan to deploy the agent with native WhatsApp Business API integration, allowing customers to place orders directly through a chat interface, and integrate physical barcode scanners for even faster manual updates.

Built With

Share this project:

Updates

Submission history