ClarityCall

Inspiration

AI voice cloning now takes under 3 seconds of public audio to produce a clone convincing enough that humans cannot reliably detect it. That gap — $2.3B lost last year to voice-based fraud — isn't a detection-algorithm problem anymore. It's a deployment problem: every serious anti-spoofing model (WavLM-Large, AASIST) is built and benchmarked for clean studio audio, then shipped nowhere near an actual phone call, where cellular codecs mangle the signal before it even reaches a detector. I wanted to close that gap: real models, running on real degraded call audio, on-device, fast enough to matter.

What it does

ClarityCall intercepts live call audio and classifies it in real time against two distinct attack surfaces:

• Synthetic speech (voice clones) — TTS/voice-conversion pipelines leave measurable artifacts in the sound wave that human vocal tract physiology cannot replicate.
• Replay attacks — playing a real recorded voice through a speaker into a mic introduces detectable room/loudspeaker signatures absent from a live human voice.

Detection and classification complete in under 200ms, and the call is cut before a scripted scam can get to the ask. The result is shown on a live threat-gauge dashboard — no technical literacy required to understand "this call is dangerous, hang up."

How I built it

The pipeline is deliberately unglamorous where it needs to be and precise where it counts:

• Audio normalization to correct for cell-network compression before feature extraction — this is the step most detectors skip, and it's why they fail outside the lab.
• Feature extraction + anomaly detection via WavLM-Large and AASIST, exported to ONNX and run through WebAssembly, entirely client-side.
• Frontend: React/Vite/Tailwind v4 for the threat-gauge UI.
• Backend: FastAPI, used only for model serving/updates — never for call audio, which never leaves the device.

I initially built this against mocked inference to validate the pipeline architecture, then did the harder work of swapping in genuine on-device WavLM-Large/AASIST inference — no shortcuts on the detection layer once the plumbing was proven.

Challenges I ran into

Training on clean datasets gets you a demo that works in silence and fails on a real call. I intentionally degraded training audio to simulate cellular compression artifacts, accepting a harder, less flattering validation curve in exchange for a model that holds up on the audio conditions it'll actually face.

The 200ms latency budget also ruled out anything that wasn't genuinely on-device — a cloud round-trip alone can eat that budget before inference even starts. ONNX/WASM on-device execution wasn't a privacy nice-to-have, it was a hard technical constraint that happened to also solve privacy for free.

Why it scales

Because inference runs entirely on-device, there's no cloud compute cost that grows with adoption — the millionth user costs the same as the first. The WASM/ONNX architecture isn't locked to one OS or carrier, so it can move into browser extensions, mobile apps, or carrier-level call handling without a rebuild. And because detection is physics-based (spectral and playback artifacts inherent to how synthetic or replayed audio is generated) rather than pattern-matched against known scam scripts, it doesn't need constant retraining against an ever-shifting scam playbook the way a text/keyword-based fraud filter would.

What I learned

Privacy and latency turned out to be the same constraint wearing two names — solving for on-device execution solved for both at once.

What's next for ClarityCall

Broader accent/language coverage for the detection model, and exploring carrier-level integration so protection doesn't depend on a standalone app being installed before the call comes in.

Built With

• digital-signal-processing
• edge-ml
• javascript
• machine-learning
• onnx
• python
• pytorch
• react
• tailwind-css
• vite
• wavlm
• web-audio-api
• webassembly