KiranaVoice

KiranaVoice: Bolo, Becho, Hisaab Rakho — The Voice-First Khata for 13M Indian Retailers

Comment

Inspiration

If you watch a neighborhood kirana owner at 7 PM during rush hour, you'll immediately see why existing fintech apps fail them. They have one hand on a digital scale, another bagging lentils, and two customers shouting orders across the counter.

In that chaos, nobody has the time to unlock a phone, tap through three menus, and type numbers into an app. Shopkeepers stick to what is fastest: a dog-eared paper diary with a ballpoint pen tied to a string, or simply trying to remember debts in their head. By the time the shutters close, half the credit (udhaar) is forgotten, calculation errors slip in, and awkward payment disputes follow ("Maine kab liya tha?").

We built KiranaVoice around a simple principle: don't force a busy shopkeeper to become a data-entry operator—make the technology adapt to how they already talk and work.

What it does

KiranaVoice is a voice-first AI ledger that turns natural, rapid Hinglish speech into structured bookkeeping and live inventory updates in under 1.5 seconds—completely hands-free.

  • Zero-Touch Voice Logging: The shopkeeper simply speaks naturally ("Ramesh ko 2 packet Maggi aur aadha kilo chawal diya, ₹120 udhaar"), and the system extracts the customer, items, quantities, and payment status instantly.
  • Colloquial Unit Normalization: Automatically standardizes regional Indian measurement terms into exact metric database values (ek pau → 250g, aadha kilo → 500g, sawa kilo → 1.25kg, peti → unit count).
  • Split Bills & In-Flight Corrections: Handles split payments ("₹100 cash diya aur ₹50 udhaar") and allows one-sentence voice fixes ("Nahi, 2 nahi 3 packet") without resetting the transaction.
  • Automated 3-Day Recovery: Automatically queues a polite, vernacular WhatsApp khata statement paired with a direct one-tap UPI payment link (upi://pay) after three days of unpaid credit.
  • Hands-Free Balance Queries: The shopkeeper can simply ask aloud, "Ramesh ka kitna baaki hai?", and hear the verified balance read back instantly through spoken audio feedback.

How we built it

  • Speech & Entity Extraction Pipeline: We used the Web Audio API to capture speech, feeding transcriptions into an LLM configured with strict JSON Schema function calling to enforce deterministic transaction objects.
  • Counter Ergonomics & Frontend: Built with Next.js, React, and Tailwind CSS, using high-contrast typography, large touch targets, and sunlight-readable palettes tailored for physical store counters.
  • Zero-Setup Cold Start Catalog: Pre-seeded a local catalog of the top 200 Indian grocery SKUs (Maggi, Parle-G, Aashirvaad Atta, Surf Excel) with baseline MRPs and colloquial aliases, eliminating days of manual data entry.
  • Automated Settlement Links: Integrated deep-link dynamic UPI intent generation directly into localized WhatsApp message templates to turn passive payment reminders into immediate settlements.

Challenges we ran into

  • Taming Hinglish & Indian Retail Slang: Counter speech blends Hindi, English numbers, brand nicknames, and colloquial weights in single sentences ("dedh sau cash, baaki udhaar"). We iteratively refined prompt boundaries and JSON schemas to normalize non-metric units reliably without hallucination.
  • The Sub-2-Second Speed Threshold: A shopkeeper will not wait for a loading spinner while a queue builds up. We heavily optimized our payload pipeline to ensure audio transcription, entity extraction, and audio confirmation happen in under 1.5 seconds.
  • Split-Tender State Reconciliations: Parsing sentences containing both immediate cash inflows and deferred credit required careful backend data modeling to ensure physical cash drawer figures and customer ledgers balanced simultaneously.

Accomplishments that we're proud of

  • Completely eliminated manual typing—a shopkeeper can speak a complex multi-item split transaction and have it logged before they finish packing the customer's bag.
  • Built a dedicated "Simulate: +3 Days Passed" test trigger on the dashboard to reliably showcase the automated WhatsApp recovery flow live during hackathon judging.
  • Designed an interface that non-technical store owners can operate comfortably without any formal onboarding or training.

What we learned

Designing software for Bharat is not about translating standard SaaS buttons into Hindi. It is about understanding the physical constraints of a busy shop. In a noisy, fast-moving retail environment, spoken audio confirmations matter far more than visual analytics, and zero-touch voice input is the only interface fast enough to keep up with daily business.

What's next for KiranaVoice

  • Money-Out & Expense Logging: Voice tracking for daily shop overheads ("Bijli bill ₹800", shop rent, helper tea).
  • Supplier & Distributor Khata: Accounts payable tracking and voice-activated inwarding when new stock shipments arrive.
  • On-Device Local AI: Quantizing the speech-to-text and parsing models to run locally on mobile hardware for 100% offline, zero-latency reliability.
Share this project:

Updates

Submission history