Inspiration India's lending market is incredibly diverse yet fragmented. With over 1.4 billion people speaking 22+ official languages across varied economic backgrounds, accessing optimal loan information remains a significant barrier. I witnessed firsthand how language limitations and financial literacy gaps prevent millions from making informed borrowing decisions. This inspired me to build a solution that democratizes loan access through AI and voice technology. What it does Dhan_Mitra is an AI-powered platform that helps Indian borrowers find and compare optimal loan terms across multiple lenders. The system features multilingual voice interface supporting natural conversation in 13+ Indian languages (Hindi, Telugu, Tamil, Bengali, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia, Assamese, Urdu, Sanskrit), intelligent rate comparison with real-time analysis across personal loans, home loans, vehicle loans, and business loans, regional optimization that accounts for state-specific regulations and local market conditions, and AI-driven personalized recommendations based on credit profile, income, and financial goals. How we built it We designed a microservices-based architecture with separate services for voice processing, loan analysis, and user management. The ML pipeline integrates Claude/GPT-4 for conversational AI and financial analysis, with fine-tuned models for Indian financial terminology and regional contexts, implementing RAG (Retrieval-Augmented Generation) for real-time lender data. The voice system uses a multi-stage pipeline: Speech-to-Text (Whisper API) → NLU → Response Generation → Text-to-Speech (Azure/Google TTS), with custom pronunciation models for Indian languages and dialect detection. The backend is built with Python FastAPI for high-performance async operations, handling concurrent voice sessions and loan calculations. The frontend uses React with responsive design optimized for Indian mobile networks. Infrastructure is deployed on Azure with auto-scaling, processing 50K+ daily decisions with <200ms latency. Challenges we ran into Building accurate multilingual NLP across 13 languages required extensive testing and custom training data, with code-switching (mixing languages mid-conversation) being particularly challenging. Indian lending varies dramatically by state, lender type, and borrower profile, so normalizing data from 100+ lenders while maintaining accuracy required sophisticated ETL pipelines. Optimizing voice quality for 3G/4G networks in tier-2/tier-3 cities meant implementing aggressive audio compression while maintaining clarity. Navigating RBI guidelines, data privacy laws, and regional lending regulations across different states added significant complexity. Accomplishments that we're proud of We achieved 95% accuracy in multilingual intent recognition, support for 13 languages with natural voice interaction, sub-2 second response time for loan comparisons across 100+ lenders, successful integration with credit bureaus for real-time profile analysis, and built scalable architecture handling 50,000+ daily decisions with high availability. What we learned We gained deep expertise in multilingual NLU and the nuances of Indian language processing, production ML deployment patterns for latency-sensitive applications, financial domain modeling and Indian lending market dynamics, voice interface design principles for non-technical users, and distributed systems architecture for high-availability financial services. What's next for Dhan_Mitra We're developing a Credit Builder Feature with personalized recommendations to improve credit scores, expanding our Lender Network through direct API partnerships with banks and NBFCs, creating Financial Literacy Modules with interactive education in regional languages, building an EMI Calculator & Simulator for advanced repayment planning, launching native Mobile Apps for Android/iOS, and establishing Regional Partner Integration through collaboration with local financial advisors and agents.
Built With
- azure-app-service
- azure-cognitive-services
- azure-functions
- docker
- fastapi
- google-cloud-speech-to-text
- kubernetes
- langchain
- openai-gpt-4/claude-api
- pinecone-vector-db
- postgresql
- python
- react
- redis
- supabase
- tailwind
- typescript
- whisper-api

Log in or sign up for Devpost to join the conversation.