Audimus

alert
protected
thumbnail
benchmark

Inspiration

Scam calls are usually caught after the fact, if at all. Truecaller and similar apps flag a number by matching it against a crowdsourced cloud database built from other users' contacts and call logs, which means the app needs your contacts to work and only helps once a number has already been reported by someone else. We wanted to flag the call itself, from what's actually being said, on the phone, with nothing shared to build that protection.

Audimus already existed as an on-device scam detector before this challenge. For Arm Create, we rebuilt its inference path to actually run its Gemma model end to end on the target hardware (previously only a keyword fallback was testable) and measured what Arm's on-device acceleration is really worth on that hardware.

What it does

Audimus reads the live transcript of an ongoing call, already produced on-screen by a captioning app like Pixel Live Caption, and reasons over it with Gemma 4 E2B running entirely on the phone. It runs two checks: an instant rule-based pass that fires the moment a high-risk phrase appears, and a Gemma pass that confirms or downgrades that flag seconds later with fuller reasoning. Every warning cites the specific phrase that triggered it, for example "caller requests an OTP code", never a free-text explanation invented after the fact.

No audio is recorded. No network call is made. Nothing about the call, the number, or the transcript ever leaves the device.

How we built it

App: Kotlin and Jetpack Compose, with an Android accessibility service reading the on-screen transcript and Gemma 4 E2B running on-device through Google's LiteRT-LM runtime.
Benchmark: cross-compiled llama.cpp for arm64-v8a with Arm KleidiAI enabled, pushed the resulting llama-bench binary and model to a Pixel 9a over adb, and ran F16, Q8_0, and Q4_0 side by side directly on the device. Q8_0 through KleidiAI's int8 kernels measured 12.6x faster prompt processing than the unaccelerated F16 baseline, on the app's actual target chip (Tensor G4, which has no SME2).
Quantization search: a separate on-device search comparing Optuna/TPE, QPSO, and QIEA against a random-search control for picking per-layer quantization strategy, since the phone's own idle time was available to spend on real trials instead of estimates.

Challenges we ran into

Tensor G4 doesn't support SME2, so we scoped the benchmark claim to what KleidiAI's standard int8 kernels actually deliver on this chip, rather than borrowing numbers from a different chipset.
A real concurrency bug: an idle-call watchdog could end a call while a Gemma analysis was still in flight for that same call, and the result would land after the session had already reset. Fixed with a transcript revision counter that discards any analysis result computed against a session that no longer matches.
adb shell truncates long inline commands, which broke passing per-layer quantization overrides as flags; moved to writing them to a file on the device instead.
Two quantization-search runs at different budgets and eval-set sizes produced different winners between the search algorithms. Rather than reporting the run that looked better, we documented both results and the open question of what caused the reversal.

What we learned

A benchmark number is only as strong as its scope. "12.6x faster" is true and defensible because it's measured on the exact target phone for the exact operation (prompt processing) that matches this app's workload shape, not a blanket claim about the whole model or every device. The same discipline applied to the Truecaller comparison: the honest claim is a difference in mechanism (content analysis on-device versus a crowdsourced number database), not an unqualified "better than."

What's next

Broader transcript source support beyond captioning apps, and extending the per-layer quantization search results back into the app's own runtime configuration now that we have real on-device data on which layers are worth the accuracy trade-off.