• 

  Weight cache analysis

• 

  Thread-count analysis

Mobile AI

Inspiration

Why should understanding technical work take an extra mental step just because I reason more comfortably in French while the tech world speaks English?

That question kept coming up while reading papers and documentation, and eventually became the starting point for this project. The problem is deceptively simple: some of the cognitive effort goes into understanding the idea; some is getting wasted by getting past the language.

Research has shown that reasoning in a non-native language can reduce accuracy (Białek et al., 2020) while processing a less-automated second language can also place additional demands on cognitive working memory (Service et al., 2002). In technical work, there's an extra wrinkle: the technical "jargon" is often the very thing you shouldn't translate : it's the common language of tech people from different nationalities.

Listen to French developers talk to each other, and you'll hear English technical jargon woven straight into French sentences, much to the dismay of the Académie française, still valiantly trying to protect Molière's language from debug, run, and quantization. C'est la vie.

That became the idea behind Tech-Explique-Moi: not to make technical jargon in English disappear, but to make technical knowledge more accessible to people who don't like to reason on complex tech concepts in English.

Tech-Explique-Moi runs Gemma 4 E2B locally on a Google Pixel 7 (Tensor G2) to turn English technical documentation into compact French arrow-schemas, a fixed semi-translation workload in "Frenglish," used purely as an instrument, not as the research subject for this hackathon. The project's actual question is whether CPU thread-count tuning and weight-cache configuration measurably change inference latency on the Tensor G2's heterogeneous cores. Every configuration is measured over 30 logged runs behind a thermal and battery readiness gate, with warm-up runs retained rather than dropped, and every correction along the way audited in the issue log.

I often read research papers on my Google Pixel 7 during my commute to work, sometimes standing on a metro platform, sometimes on the train itself. That made the use case very concrete for me: having a small LLM running directly on my phone, without relying on a server, could turn an everyday situation into a genuinely useful tool. Instead of waiting until I have access to a computer or a full development environment, I could use that idle time to understand technical material in the language I naturally think in.

That became one of the motivations behind Tech-Explique-Moi: not just to benchmark an LLM on a mobile device, but to explore whether local, serverless AI can solve a real, recurring need in everyday life, with the constraints of a device that is actually in my pocket.

The project was built and evaluated using French as the target language because French is my native language, and therefore the language in which I personally wanted to reduce this cognitive friction. But the approach is deliberately language-agnostic: the same methodology can be applied to any language a developer considers their native or most comfortable language for deep reasoning. French is simply the language I chose as the concrete case study; the underlying principle remains the same regardless of the language.

In a field where English is the default working language, this project is my modest attempt to make technical work a little more inclusive, by removing unnecessary linguistic friction without sacrificing the terminology engineers actually use. The goal is simple: make the same technical ideas easier to access, regardless of the language you feel most at ease thinking in deeply.

The engineering question grew naturally from there: could this use case be optimized so that fewer tokens, computational and cognitive, are spent on translation, and more on the work that actually matters? For me, it also means spending fewer tokens on translating technical English documents (research papers...) and more on higher-value tasks. In a world where tokens are a finite resource and compute is never free, how we spend them matters. The experiment measures the computational side; the cognitive side is the motivation behind it.

In both cases, the underlying question is the same one that started the project: how much of the available capacity is doing the work that actually matters?

What it does

Tech-Explique-Moi is an execution-level optimization study of a fixed local LLM workload on Arm mobile silicon. Gemma 4 E2B runs entirely on-device on a Google Pixel 7 (Tensor G2) via LiteRT-LM, turning English technical documentation into compact French arrow-schemas — cause→effect chains, with established technical terminology left in English.

That translation task is the fixed instrument, not the research subject. The question it was built to answer: does runtime CPU thread-count tuning measurably change inference latency on the Tensor G2's heterogeneous 2+2+4 layout (4× Cortex-A55, 2× Cortex-A78, 2× Cortex-X1) — and if it does, is the result real, or an artifact of an Android scheduler quietly deciding which cores actually run the work?

Measured answer. 4 threads is fastest: 24.874 s median end-to-end vs. 29.561 s for the runtime's own auto default, 1.188× speedup, 15.9% latency reduction, n=30 per configuration. It then survived an independent replication in a separate session (20.682 s, 1.429×): same ranking, same direction, so it is reported as ROBUST rather than as one lucky campaign.

The interesting part is that latency is not monotonic in thread count while CPU utilization is. At 1/2/4/8 threads the process draws 96.7% / 166.1% / 284.5% / 499.8% CPU, the flag genuinely engages more cores, yet 8 threads is slower than 4 (30.038 s), consistent with the A55 littles being pulled onto the critical path. 4 threads was also the tightest configuration of the sweep (SD 2.123 s vs. baseline's 3.981 s) at essentially the same total CPU as baseline: same silicon budget, lower median, half the variance.

A second lever, tested at that winning thread count: disabling XNNPACK's weight cache costs 53.5% latency (38.177 s) and nearly doubles peak RSS (4,017 MB vs. 2,231 MB). Best combination found: 4 threads with the weight cache resident in memory.

It ships with a research dashboard, all configurations side by side on latency, decode throughput, CPU utilization, peak memory, and the replication verdict, plus a "Try It" page where you can paste your own technical documentation and see the exact frozen prompt that would be sent to the model, next to a real measured output from the campaign. That page states plainly that it does not run inference: a browser cannot drive adb.

How I built it

Runtime. LiteRT-LM, built from source (litert_lm_advanced_main, Bazel 7.6.1 + Android NDK r28b under WSL2, --config=android_arm64), deployed over adb and invoked with LD_LIBRARY_PATH=/data/local/tmp.

Model selection (Phase A). 15 technical documents, Arm big.LITTLE, Linux CFS and cgroup v2, RFC 5681 congestion control, WebAssembly, Vulkan, LLVM IR, DNS, CUDA, Git internals, scored against explicit criteria and an 80% pass threshold, both fixed before a single output was scored. Result: 4.08/5 mean, 0 critical semantic errors across 36 observations, 12/15 PASS = 80.0%, clearing the pre-registered bar by exactly nothing, and reported that way rather than rounded into a win. Output was byte-identical across repeated runs of the same document on all 15 documents; that observed determinism is what made a 2-run-per-document sample defensible instead of hand-waved. The system prompt was frozen only after comparing five structurally distinct output formats (arrow-schema, table, SI/ALORS conditional, concepts+relations, guided Q&A) across 6 criteria, arrow-schema was the only one to pass all six.

Measurement (Phase B). One frozen input, hash-locked (SHA256 7edcb837…, re-verified on-device before every session). The workload document was chosen because its Phase A score sat 0.08 from the corpus mean, deliberately not a best case. A Python harness enforces a readiness gate (Android thermal status NONE/LIGHT, battery ≥50%) before every run, launches the binary, polls /proc//stat and /status once per second for CPU utilization and peak RSS, times end-to-end latency, and writes every run to CSV. Each of the 5 thread configurations had to meet a stabilization criterion (rolling 5-run median within 10%) before its 30 valid runs began; warm-up runs are retained and labelled, never silently dropped. Configurations were interleaved rather than run in a fixed order, with 60 s recovery between batches.

Device isolation is a session-level precondition, not a courtesy: runs happen on battery over wireless ADB (charging plus CPU-bound inference is two independent heat sources), Airplane Mode with Wi-Fi re-enabled on top, auto-updates off, no physical interaction with the device.

Challenges I ran into

Full audit in docs/ISSUE_LOG.md. The pattern in all of them is the same: the device is the source of truth, not the documentation and not the terminal.

Assumptions that had to be corrected empirically. MediaPipe's LLM Inference API is in maintenance-only mode, redirecting the project to LiteRT-LM. The first binary silently ignored --max_output_tokens, five consecutive runs produced exactly 920 tokens regardless of the flag; grep on the source showed that entry point never reads the length-control settings at all, forcing a switch to litert_lm_advanced_main. Then --benchmark_prefill_tokens turned out to activate a synthetic fake-token mode that discards the real prompt (should_print_output = benchmark_prefill_tokens == 0), which is why a benchmark run produced 0 prefill and 0 decode turns. XNNPACK is not an independently togglable CPU path, verified in the runtime's source, not inferred from a flag list, so --backend=cpu is the XNNPACK path and there is no non-XNNPACK baseline to compare against. The reference device was initially misidentified (Pixel 7a vs. the actually connected Pixel 7), caught via getprop before it could propagate through every downstream document.

The 11× variance that became a lever. Two nominally identical runs of the "default" configuration returned 10,769 ms and 941 ms Init Executor, an uncontrolled state dependency, almost certainly a stale on-disk compilation cache. Rather than average over it, both cache flags were pinned explicitly on every single Phase B run, and the cache itself was promoted to a measured lever. Source verification later confirmed the mechanism: litert_compiled_model_executor_utils.cc passes the path straight to SetXNNPackWeightCachePath(), so it is a compiled-kernel cache, not a model-load optimization, which explains why disabling it degrades decode throughput and not just startup.

Accomplishments that I'm proud of

A result that is falsifiable and survived the falsification attempt. The winning configuration was re-run in a separate session under the same protocol, against a decision rule written before the replication (ranking must hold, both speedups on the same side of 1.0×). It held, and it is reported alongside the caveat that limits it: without CPU affinity, thread-count results on a heterogeneous 2+2+4 CPU mix the configured parameter with unobserved scheduler placement. That caveat appears first in every result section, not in a footnote.

And the whole pipeline is reproducible from the repo: frozen prompt with a published hash, raw per-run CSVs including warm-up runs, statistics computed by script, and a dashboard generated from those CSVs rather than transcribed into slides.

What we learned

Runtime documentation can be wrong or stale in ways only source reading catches, XNNPACK's coupling to the CPU backend, and the weight cache's true nature as a compiled-kernel cache, both came from the runtime's own source, not from a flag reference.

CPU heterogeneity on a mobile SoC is not a nuisance to optimize away with a cleaner untested lever; it is a confound to name explicitly and control for with replication. And utilization is not progress: 8 threads burned 500% CPU to finish slower than 4 threads at 284%.

What's next for Tech-Explique-Moi

Directions set aside for documented reasons:

CPU affinity — the only lever that would fully resolve the heterogeneity confound, but it means driving the Android scheduler directly, outside this project's runtime-exposed-parameters-only scope. Full 5×2 factorial — the two levers were run sequentially, with the cache effect spot-checked at both the winning and losing thread counts rather than measured across the whole grid. A real interaction would show up there first. --enable_ynnpack — a CPU delegate that runs "before XNNPACK," present in 13 places in the source, off by default and publicly undocumented. Worth measuring once it is documented. i8mm / KleidiAI — verified unavailable on Tensor G2: /proc/cpuinfo shows no i8mm or SVE flags, and the cores top out at Armv8.4-A dot product where I8MM needs Armv8.6-A. Directly relevant on newer Armv9 silicon. More Arm devices — the current result is intentionally scoped to one Pixel 7, not generalized. Running the same frozen protocol on a second SoC is the cheapest way to learn whether the 4-thread optimum is about this topology or about the runtime. Quantization, pruning, distillation — orthogonal to execution-level tuning, and each would require re-validating output quality from zero. Not a techie optimization but is related to our use case : A direct study of cognitive load itself — reading time and comprehension accuracy on arrow-schema output versus raw English source, with bilingual readers — is the natural extension of the original inspiration.

And with the configuration now established, turning the prototype into a native Android app: share or paste a paper, run the model locally at the measured-optimal settings, get the compact explanation with no server and no connection. That closes the loop between the engineering experiment and the everyday use case, a local LLM that is useful precisely because it is always with you.

Built With

• androidndk
• bazel
• gemma4e2b
• googlepixel7
• linux
• litert-lm
• wsl2