NammaCare — Multilingual Civic AI

Inspiration

Civic complaint systems quietly exclude the people who need them most. They assume you speak English, own a smartphone, and can navigate a web form — and then they go silent after "submitted." In a city like Bengaluru, where a single street corner might be described in English, Hindi, or Tamil, that design leaves millions unable to report the pothole outside their home or the garbage pile that won't get collected.

We wanted to flip that assumption. What if reporting a civic issue were as easy as answering a phone call or typing one sentence in your own language — and what if the citizen could later see a photo of the fixed problem, not just a status word?

That became NammaCare: two independent ways in (text and voice), one shared case system, and a closed loop of report → triage → resolution proof → citizen tracking.

What it does

• Text agent — a LangGraph state machine understands a complaint, insists on a real location, and requires explicit confirmation before storing anything.
• Voice agent — an authorized outbound call answered by a native Gemini Live speech-to-speech assistant, in English, Hindi, or Tamil, with noise suppression and a live safety guardrail.
• AI triage — every case is scored for sentiment, urgency (1–10), and community impact; open cases are grouped into "emerging hotspots" with one-line action briefs; and duplicate reports are caught at intake so operators see signal, not noise.
• Proof loop — citizens attach a before-photo; operators upload an after-photo; a human approves the AI's comparison before the case resolves; the citizen sees the approved completion photo on a public tracking page.

How we built it

The backend is FastAPI + LangGraph, with SQLite as the shared case store. The text conversation is modeled as an explicit state graph — understand → ask_problem → ask_location → ask_confirmation → check_duplicate → create_case — so the confirmation-before-storage rule and the duplicate-detection branch are structural, not prompt-hoped-for.

The voice path is a Pipecat pipeline: Exotel places the call, RNNoise + Silero VAD clean the 8 kHz audio, Gemini Live handles speech-to-speech end-to-end, and an AWS Bedrock guardrail screens civic claims while audio streams.

Every AI touchpoint has a fail-open design. The clearest example is text intake, where we care most about cost. It runs on a three-tier chain:

$$ \text{Qwen3-235B (Bedrock)} \;\rightarrow\; \text{Gemini 3.5 Flash} \;\rightarrow\; \text{deterministic rules} $$

If Qwen errors or times out, the same request is silently retried on Gemini; if that fails too, a built-in rule-based classifier answers. The citizen never sees a blank.

The cost decision. Text intake is the highest-volume LLM call — it fires on every chat turn — which made it the right place to introduce an open-source model. Moving it from Gemini 3.5 Flash to Qwen3-235B on AWS Bedrock (per 1M tokens, provider-published):

Model	Input	Output
Gemini 3.5 Flash	\$1.50	\$9.00
Qwen3-235B (Bedrock)	\$0.1545	\$0.618

The reduction:

$$ \text{input: } 1 - \frac{0.1545}{1.50} \approx 90\%, \qquad \text{output: } 1 - \frac{0.618}{9.00} \approx 93\% $$

For a rough per-turn cost — with $n_{\text{in}}$ input and $n_{\text{out}}$ output tokens per turn and prices $p_{\text{in}}, p_{\text{out}}$ per token:

$$ \text{cost}{\text{turn}} = \frac{n{\text{in}} \cdot p_{\text{in}} + n_{\text{out}} \cdot p_{\text{out}}}{10^6} $$

Across the intake path that works out to roughly ~10× cheaper per turn — the difference between a demo and something a city could afford at thousands of complaints a day. We kept the switch honest: only the high-volume call moved. Voice (Gemini Live) and the lower-volume scoring, summary, and photo-analysis calls stayed on Gemini.

Challenges we ran into

• Structured output on Bedrock. Bedrock's .converse() has no native schema mode like Gemini's response_schema. We solved it with prompt-instructed JSON parsed through the same _sanitize_analysis safety net the Gemini path already used — so a malformed Qwen response couldn't corrupt conversation state.
• Privacy without losing observability. We wanted rich diagnostics (which model served each turn, latency breakdowns) without ever logging complaint text, phone numbers, or keys. The result: hashed session tags, phone masking in access logs, exception-class-only errors, and a Server-Timing header that proves qwen;dur=… live in the browser.
• Human-in-the-loop resolution. AI photo comparison is genuinely useful but must never auto-close a case. We made approval a mandatory human step, and blocked evidence-backed cases from skipping proof review via the ordinary status endpoint.
• Keeping the demo un-brittle. Vendored Leaflet (no CDN), an offline landmark table so the map works with no network, and a rate-limited, fail-open geocoder — so a flaky conference Wi-Fi can't break the live demo.

Accomplishments that we're proud of

• A genuinely inclusive front door. Two channels (text + voice) across three languages, feeding one shared case system — a citizen with no smartphone and no English can still report an issue by phone.
• A ~10× cheaper intake path with no reliability loss, because the Qwen→Gemini→deterministic fallback protects every single turn.
• A closed loop most systems never finish — report, triage, AI-assisted resolution proof with a human approver, and a public tracking page showing the citizen the actual fixed-problem photo.
• 83 passing tests that never touch a live provider — the whole intake fallback chain, duplicate detection, photo sanitization, and resolution-proof workflow are verified with every provider stubbed.

What we learned

• State machines beat prompts for safety. Putting "confirm before storing" and "check for duplicates" into the graph structure made them provable — and testable — instead of hoping the model complied.
• Fail-open is a feature, not an afterthought. Every provider (Qwen, Gemini, the guardrail, geocoding, enrichment) degrades gracefully. That single principle let us adopt a cheaper, newer model without betting reliability on it.
• The cheapest place to optimize is the highest-volume call. We resisted the urge to move everything to open-source and instead moved the one call that dominates cost — a smaller change with most of the savings.
• Deterministic grouping + generative phrasing is a great division of labor. Our hotspot clustering matches (category, area) in plain Python (exact, testable) and only asks Gemini to phrase the summary. Cheaper, more reliable, and no hallucinated groupings.

What's next for NammaCare

Live voice transcript on the dashboard, WhatsApp as a third intake channel, semantic (embedding-based) duplicate detection, and moving the remaining analysis calls to open-source models on the same fail-open pattern.

Built With

• amazon-web-services
• gemini
• langgarph
• pipecat
• python
• qwen