iTantra

iTantra enables offline, cross-language communication between phones using Wi-Fi Direct, with voice translation, lightweight text transfer, and one-tap SOS alerts for emergencies.

Comment

iTantra

Inspiration

When disasters strike or teams work in remote areas such as mountains, mines, or field stations, the first thing to fail is connectivity. Even when phones still work, India’s linguistic diversity adds a second barrier: a rescuer and a person in distress may not share a language. Cloud-based speech and translation APIs need the internet, and sending raw voice over weak, low-bitrate links is unreliable.

We wanted to answer one question: What if two phones could talk to each other in their own languages, with no internet, no towers, and no cloud?

That became iTantra, our response to the problem statement on multilingual TTS and STT for low-bitrate links.

What It Does

iTantra is a fully offline Android app with a multilingual interface. It works like this:

  1. Speak: The sender talks into their phone.
  2. Convert: On-device speech-to-text turns speech into text, and the language is detected.
  3. Send: The text is compressed into tiny packets and sent directly to the nearby phone over Wi-Fi Direct / Bluetooth.
  4. Hear: The receiver’s phone decompresses the message, translates it if needed, and speaks it aloud in the receiver’s own language.

It also has a non-verbal SOS button that broadcasts a BLE beacon without requiring any audio. Active iTantra devices nearby detect it automatically.

How We Built It

Sender Side

  • Silero VAD (about 1 MB) listens continuously at near-zero CPU usage and wakes heavier models only when speech is detected.
  • AI4Bharat IndicConformer converts speech to text for 10 Indian languages.
  • A lightweight language model rescoring step uses context to fill in partially detected words.
  • The text is compressed and converted into packets.

Transmission

  • Wi-Fi Direct enables peer-to-peer connections without a router or internet.
  • CSMA/CA in half-duplex mode helps avoid packet collisions.
  • Text-only packets, with lightweight retransmission and FEC, keep bandwidth requirements minimal.

Receiver Side

  • The message is decompressed, and the receiver’s language is identified.
  • If the languages differ, the text is translated into the receiver’s language.
  • AI4Bharat Indic-TTS converts the message into speech.

Making It Run on Ordinary Phones

  • INT8 quantization with TensorFlow Lite makes models approximately 4× smaller and 2–3× faster on low- and mid-range CPUs.
  • Our DeepSeek-inspired offloading approach works like an LRU cache. Only the STT/TTS model for the active language stays in RAM, while the remaining models stay in flash storage until the language is switched.
  • The NNAPI delegate accelerates on-device inference.
  • The app uses native Kotlin + Android SDK for direct microphone and audio access.

Challenges

  • Unreliable, low-bitrate links: We solved this by never transmitting audio, using only compressed text packets with retransmission and FEC.
  • High end-to-end latency: The speech → text → send → speech pipeline has multiple stages. INT8 quantization and on-demand model loading help reduce delays.
  • Limited RAM and CPU on phones: Running multiple language models simultaneously is impractical on budget devices, leading us to develop our offloading and LRU-style paging design.
  • Accuracy on Indian languages and noisy speech: We use per-language models and context-aware correction to improve partial or noisy transcriptions.
  • Battery drain: Always-on listening can be expensive, so we use VAD gating to keep idle CPU usage near zero.
  • Packet collisions when devices transmit simultaneously: CSMA/CA helps manage channel access and reduce collisions.

Accomplishments That We're Proud Of

  • Designed a complete offline voice-to-voice pipeline (speech → text → packets → translation → speech) that requires no internet and incurs zero recurring cloud costs.
  • Using AI4Bharat's Indic-trained models, we're targeting a ~12–15% word error rate (WER) on Indian languages, aiming for competitive accuracy while remaining fully offline.
  • Built a smart memory strategy using DeepSeek-style offloading to keep only one language's models in RAM.
  • Added a non-verbal BLE SOS so distress signals can work even when someone cannot speak.
  • Made the solution accessible: voice alerts help users who cannot read text warnings, including non-literate and differently abled users.
  • Kept the solution open-source and affordable, making large-scale deployment more realistic.

What We Learned

  • Sending text instead of audio is the single biggest improvement for low-bandwidth links. The payload shrinks from kilobytes per second to bytes.
  • Model compression matters as much as model accuracy. INT8 quantization helped make the application practical beyond flagship phones.
  • Memory management is a design problem. Treating language models like pages in an LRU cache saves RAM without compromising usability.
  • Effective VAD gating is a simple way to save battery in always-listening applications.
  • Indian-language speech recognition and TTS need language-specific models. One-size-fits-all models may not provide sufficient accuracy.
  • Standard Android APIs such as WifiP2pManager let us focus on efficiency and user experience instead of reinventing networking.

What's Next for iTantra

  • Scale beyond 10 languages toward supporting all 22 officially recognized Indian languages.
  • Multi-hop relay: Allow messages to travel across several nearby phones, extending communication range during disasters.
  • Group and broadcast messaging: Enable a responder to alert an entire team or area.
  • Domain-tuned alert voices and vocabulary for disaster response, police, railways, mining, and utilities.
  • Stronger noise robustness for stressed, outdoor, and emergency speech.
  • Field trials with disaster-response teams, remote stations, and emergency departments.
  • Further battery and latency optimization, including streaming and sentence-chunked TTS playback.

Built With

Share this project:

Updates

Submission history