Awaaz

Awaaz: DeepFake Audio Detection

Inspiration

Almost half of Indian adults have faced, or know someone hit by, an AI voice-clone scam, and around 70% say they can't tell a clone from a real voice (McAfee, The Artificial Imposter). A few seconds of your voice from Instagram is enough to clone it. Then your mother gets the call: "Mummy, mera accident ho gaya, jaldi paise bhejo, papa ko mat batana." An Indore school teacher lost ₹97,000 this way.

Most detectors answer "AI or human?" for clean English audio. Real scams arrive as compressed WhatsApp notes and phone calls in Hindi, Tamil, Telugu or Hinglish, and a worried parent's real question is: is this really my child, or a scam?

What it does

Upload a voice note or call recording. Awaaz runs three independent checks:

Synthetic voice: does it sound machine-made? It scores every 4-second window and shows the scores as a timeline.
Speaker match: does it sound like a family member you enrolled? Enrolled voices stay only in your browser session.
Scam script: Whisper transcribes and translates, then highlights urgency, secrecy, money/UPI, OTP and police cues in Hindi, Hinglish and English.

The checks combine into a traffic light. 🔴 means "Likely a cloned voice of Mum: call her back on her saved number." 🟠 means "Sounds human, but follows a scam script." 🟢 means "No warning signs", never "verified real".

How we built it

Data: IndicSynth clones (FreeVC24, XTTS-v2) against real Kathbath speech in 7 Indian languages. Splits are speaker-disjoint across both classes, so a test voice is never heard in training, either as itself or as a clone. Telugu and Malayalam are held out entirely.
Real-world channels: most training clips pass through randomised WhatsApp Opus, phone-line, MP3/AAC, reverb and noise simulation.
Detector: frozen wav2vec2 XLS-R (300M) with a learned mix of its layers and an attentive statistics-pooling head, compared against a log-mel ResNet baseline:

$$h = \sum^{L}_{l=1} \alpha_{l} \, h^{(l)}, \quad \alpha = \operatorname{softmax}(w)$$

A clip is split into 4 s windows, and its score is the mean of the top third of window scores, so a short synthetic stretch isn't averaged away. The decision threshold is calibrated on held-out speakers at the equal error rate:

$$\text{FAR}(\tau^{\ast}) = \text{FRR}(\tau^{\ast}) = \text{EER}$$

Speaker match: ECAPA-TDNN embeddings (SpeechBrain), cosine similarity against enrolled voices:

$$\text{sim}(a, b) = \frac{a \cdot b}{\lVert a \rVert \, \lVert b \rVert}$$

Infrastructure: training on Kaggle (2× T4), with checkpoints and results versioned on the Hugging Face Hub. The demo is a Gradio app on Spaces (ZeroGPU), backed by an automated test suite.

Results (speakers never heard in training)


Error rate on clean, WhatsApp and MP3 audio	~1% (~2% over phone lines)
Error rate on Telugu and Malayalam, never seen in training	under 1%
Accuracy on voices outside all training data	~93% (every real family voice note called real)
"Cloned voice of " fired on the real person	never
Scam-script check (Hindi / Hinglish / English)	~93% F1

Challenges we ran into

Is the model cheating? A near-perfect validation score made us suspicious. Real and fake clips differed in two ways unrelated to the voice: padded digital silence and the 7–8 kHz band. We removed each at test time with dithering, a 7 kHz low-pass, and both together. Accuracy held, so the model isn't relying on dataset quirks.
Re-recorded audio: a clone played on one phone and recorded on another is the hardest case, at ~18% error. Adding simulated re-recordings to training is the next step.
Unseen cloning tools: a model trained without XTTS-v2 drops to ~25% error on it. We report that instead of hiding it.
Deployment: ZeroGPU pickles a GPU function's arguments, and XLS-R can't be pickled, so the first real request crashed. We now load the model once at startup and pass only audio arrays. If a visitor's GPU quota runs out, the check falls back to CPU instead of failing. Both cases have regression tests.
Whisper on low-resource languages: Whisper sometimes misdetects Telugu and hallucinates a translation. The voice verdict doesn't depend on the transcript, but it's a clear next fix.

What we learned

99% on your own test split means little. The numbers that matter come from held-out speakers, unseen languages, messy channels and real voice notes from people around you. A scam detector's wording also matters as much as its model: it should tell people what to do (hang up, call back on a saved number), not just output a probability.

What's next

Simulated re-recordings in training, language-confidence gating for transcripts, more cloning tools, a WhatsApp bot so families can forward a suspicious note directly, and on-device inference for privacy.