Janavaani

Inspiration

In a diverse country like India, citizens often raise complaints through voice calls in their native languages. Unfortunately, many of these voices go unheard due to language barriers, manual systems, and a lack of intelligent prioritization. We wanted to build a system that listens to every citizen — regardless of language — and helps city officials respond faster and more effectively.

What it does

Janavaani is a multilingual AI-powered voice assistant that: Converts citizen voice complaints into text using speech-to-text Detects the spoken language and translates it into English Analyzes sentiment and emotion to identify urgency Classifies complaints into categories like sanitation, electricity, water, etc. Helps government officials view, prioritize, and respond to critical issues It acts as a digital bridge between people and governance — turning raw voice input into actionable insights.

How we built it

We used: Whisper ASR for accurate speech-to-text transcription LangDetect for language identification MarianMT (Hugging Face) for multilingual translation Transformers & LSTM models for emotion and sentiment analysis Scikit-learn / Hugging Face pipelines for issue classification Python with a modular architecture for easy scaling Sample audio complaints to simulate real-world urban service calls All tools and models used were open-source and optimized for real-time performance.

Challenges we ran into

Handling short, noisy, or emotional speech inputs during STT Language detection accuracy for similar-sounding languages Finding publicly available multilingual training datasets Real-time translation model loading and memory optimization Balancing simplicity with intelligence in a single system

Accomplishments that we're proud of

Created a fully working multilingual pipeline from voice to issue classification Successfully detected and translated Telugu, Hindi, and English inputs Integrated sentiment detection to prioritize urgent complaints Built a solution that is both socially impactful and technically scalable

What we learned

Building smart city solutions requires empathy and cultural understanding, not just code Combining multiple open-source AI models can lead to powerful outcomes Language diversity in India makes AI voice understanding uniquely challenging — and rewarding

Emotion-aware systems significantly improve how technology serves citizens

What's next for Janavaani

Add voice response generation in regional languages Deploy on city helpline systems or WhatsApp for public use Integrate a visual dashboard for city officials with real-time updates Train custom models for better local dialect understanding Partner with municipal bodies to pilot Janavaani in live environments

Built With

• huggingface
• langdetect
• marianmt
• openai
• python
• scikitlearn
• transformers
• whisper