DukanMitra

A Telugu-first AI shop assistant for small kirana retailers to manage stock, sales, and credit.

Comment

Inspiration

Small kirana shop owners often struggle to manage stock, sales, profit, and customer credit (udhar) recorded across paper notebooks. I wanted to build a voice-enabled, Telugu-first AI assistant that allows local retailers to query shop data naturally in their own language and manage day-to-day transactions with human-in-the-loop safety.

What it does

DukanMitra combines a Telugu/English conversational assistant with a visual shop dashboard:

Natural Language Queries: Answers inventory stock, pricing, sales, profit, and customer credit questions in Telugu or English.

Safe Conversational Sales: Prepares server-stored sale proposals with full details (item, quantity, total price, stock deduction, and credit impact) requiring explicit owner confirmation before updating records.

Shop Analytics Dashboard: Displays revenue trends, best-selling items, low-stock warnings, and customer credit-ageing alerts (30+ days overdue).

Supplier Bill Scanner: Extracts items and quantities from supplier bills using Gemini vision for quick review and stock entry.

MCP Server: Exposes 20 shared business tools over a self-hosted Model Context Protocol (MCP) server for external tool clients.

How we built it

Backend: Python and Django handling business logic, session-bound approval flows, and database transactions.

AI Agent: Google Gemini with function calling for natural language query understanding and vision-based bill extraction.

MCP Integration: FastMCP providing a Streamable HTTP MCP server exposing inventory and sales tools.

Frontend & Voice: Responsive HTML5/JavaScript dashboard with browser Web Speech APIs for Telugu voice input and speech synthesis, packaged as an installable PWA.

Deployment: Hosted on Render with Gunicorn and SQLite for live demonstration.

Challenges we ran into

Model Access & Fallback: Amazon Bedrock model invocation was blocked on my AWS account, requiring a fast pivot to Google Gemini to keep development moving.

Transactional Safety: Preventing the LLM from hallucinating or executing sales directly without human approval. We designed a session-bound, server-stored proposal mechanism that verifies fresh business data before executing.

Vernacular Language Handling: Ensuring Telugu speech recognition, transliteration, and customer aliases mapped accurately to underlying database records.

Accomplishments that we're proud of

Built a complete, end-to-end working assistant for kirana retailers solo from scratch.

Implemented a secure, human-in-the-loop sale proposal and confirmation architecture with automated safety tests.

Designed a dual-interface model sharing business logic across both a web agent and a standalone 20-tool MCP server.

What we learned

How to structure Model Context Protocol (MCP) servers with FastMCP.

The importance of explicit server-side state machines and session boundaries when letting conversational agents initiate consequential financial transactions.

Effective prompts and schema design for multimodal bill parsing in Indian retail contexts.

What's next for DukanMitra

Multi-tenant shop authentication and persistent PostgreSQL database.

Deep integration with UPI payment links and automated PDF receipt generation.

Expanding language support to Hindi, Tamil, and Kannada.

Real-time WhatsApp Business Cloud API integration for automated supplier orders.

Built With

Share this project:

Updates

Submission history