reconcile-and-reply
order-board
understand-voice-to-record

The Story Behind Bhasha-Procure

The Inspiration

Our journey began with a mismatch between clean software assumptions and industrial reality. While modern enterprise resource planning (ERP) systems are designed around the assumption that vendors will log into a portal and fill out clean, structured fields, the actual supply-chain ecosystem functions entirely on informal, rapid-fire messaging. Monica's grounding in supply-chain management revealed that vital procurement infrastructure regularly experiences delays simply because critical engineering data is locked inside vernacular WhatsApp voice notes, unstructured SMS threads, and multi-lingual emails. We realized that forcing vendors to change their behavior doesn't scale but building an India-centric AI stack to meet them where they are does.

How We Built It

We engineered a localized, two-layer pipeline built entirely on an open-weights architecture to preserve data sovereignty and avoid external API dependencies.

Layer 1 (Understand): To handle heavy industrial and construction jargon across 6 regional languages, we implemented IndicConformer-600M for speech-to-text processing. For the text-to-JSON extraction module, we utilized BharatGen Param 17B. We implemented a rigorous token verification layer using grammar-constrained decoding, mathematically forcing the language model's output distribution to strictly conform to a pre-defined JSON schema, completely eliminating syntactic hallucinations.
Layer 2 (Reason): The structural JSON output maps into an active state machine tracking 8 precise lifecycle states per line item. It actively runs an automated discrepancy overlay to calculate variance matrices across material specifications, budgets, and lead times.

To handle uncertainty safely, we added a validation layer using temperature scaling. The model's raw logits are scaled by an inverse temperature factor to minimize the Expected Calibration Error (ECE), defined mathematically as:

$$\text{ECE} = \sum_{m=1}^{M} \frac{|B_m|}{N} \left| \text{acc}(B_m) - \text{conf}(B_m) \right|$$

If the calibrated confidence score falls below a dynamically computed threshold (e.g., $0.85$), the record trips a gate and holds for human verification before syncing downstream to Kaya's Project Graph.

Challenges Faced

Our primary hurdle was the complete absence of domain-specific multi-lingual datasets for heavy industries like construction and precision cooling. Standard off-the-shelf automatic speech recognition (ASR) baselines routinely failed on specialized acoustic technical terms.

To overcome this, we built an internal TTS $\rightarrow$ STT data flywheel. We curated a dense, synthetic multi-lingual text corpus mapping conversational variations into schema-valid JSON pairs. We then passed these scripts through regional text-to-speech engines to generate highly specific acoustic datasets. By training on an optimized ratio of $70\text{--}80\%$ real conversational audio to $20\text{--}30\%$ entity-dense synthetic utterances, we bypassed the data bottleneck completely.

What We Learned

This project redefined how we view the intersection of research-grade AI and production-level deployment. We learned that general-purpose accuracy is a misleading metric in enterprise workflows. In a procurement pipeline, missing a single numerical entity or misinterpreting a unit shift can blow past a multimillion-rupee project budget.

This forced us to pivot away from generic evaluation benchmarks and implement hard group-disjoint partitioning by speaker and supplier during training. Most importantly, we validated our scope: instead of overcommitting to a fragile multi-lingual model across 22 languages simultaneously, a modular architecture focused on native optimization for 5–6 core languages paired with flexible parameter-efficient low-compute adapters is the most robust way to deploy software into the field.