MozhibotAI

Inspiration

India has 1.4 billion people — yet no voice AI speaks Tamil, Hindi, or Telugu offline. Siri, Google Assistant, and Alexa all require internet, send data to US servers, and barely support regional languages. Students in rural areas, patients in hospitals, and farmers in villages are left out entirely.

I wanted to build something that works for everyone — no internet, no cost, no data leaving the device. MozhiAI was born from that idea: an AI that speaks your language, on your hardware, for free.

What it does

MozhiAI is a complete offline voice AI platform for Tamil, Hindi, and Telugu, optimized specifically for Arm-powered hardware.

A user speaks in their native language → MozhiAI transcribes the speech (VaniSTT) → generates an intelligent reply (ArmLM) → speaks it back in the same language (VaniTTS). The entire pipeline runs in under 3 seconds on Arm64 hardware with zero internet.

Developers can also use the MozhiAPI REST endpoints to build their own multilingual voice apps on top of MozhiAI. A live benchmark dashboard (ArmBench) shows real-time optimization metrics.

How we built it

Stack: Python · FastAPI · React · Whisper.cpp · llama.cpp · gTTS · ngrok · Vercel

Pipeline: Voice → Whisper tiny (INT4 quantized) → Phi-3 Mini (Q4_K_M) → gTTS → Voice

Arm Optimizations applied:

INT4 Quantization — compressed Phi-3 Mini from 7,372 MB to 2,282 MB (4×)
INT4 Quantization — compressed Whisper from 244 MB to 61 MB (4×)
Arm NEON SIMD — llama.cpp compiled for AArch64 auto-vectorizes on Arm
ONNX Runtime — Arm-optimized execution provider for TTS inference

Infrastructure: Entirely free — Google Colab (AI backend) + Vercel (frontend) + GitHub (code). Target deployment: Oracle Ampere A1 (4× Arm64 cores, 24GB RAM, forever free).

Challenges we ran into

Model downloads on Colab — Piper TTS Tamil voice files returned 404 from HuggingFace. Switched to gTTS which supports Tamil natively.
Git pushing 2.5GB — accidentally committed the Python venv folder. Used git filter-branch to rewrite history and clean the repo.
Colab session restarts — every 12 hours, all packages and models reset. Solved by saving models to Google Drive and writing a one-cell restart script.
Multilingual LLM responses — Phi-3 Mini Q4 occasionally replies in English instead of Tamil/Hindi. Addressed with stronger system prompts and language-prefixed user messages.
ngrok authentication — free ngrok requires account token. Documented clearly in setup guide for other developers.

Accomplishments that we're proud of

✅ First open-source offline voice AI supporting Tamil, Hindi, and Telugu
✅ 4× model compression on both STT and LLM models via INT4 quantization
✅ Complete 5-component platform built entirely for free — ₹0 infrastructure
✅ Live public demo at arm-ai.vercel.app accessible to anyone
✅ OpenAI-compatible REST API — any developer can build on MozhiAI
✅ Measured benchmark results: 2.51 tok/s, 1841ms STT, 213ms TTS
✅ MIT licensed — fully open source and reusable

What we learned

How INT4 quantization works mathematically and why it preserves quality
How llama.cpp auto-enables Arm NEON SIMD at compile time on AArch64
How to build and expose a FastAPI server from Google Colab via ngrok
The real challenges of multilingual AI — small models struggle with non-English scripts at low bit widths
How to build a complete AI product with zero budget using free tiers

What's next for MozhiAI

Oracle Ampere A1 deployment — move from Colab to permanent free Arm64 server for always-on availability
Larger LLM — deploy Llama 3.1 8B on Arm64 for pure Tamil/Hindi responses without English mixing
More Indian languages — Kannada, Malayalam, Bengali, Marathi
Android app — native on-device inference using whisper.cpp Android bindings for true mobile Arm deployment
Arm Performix benchmarks — official Arm benchmark scores for credible performance comparison