-
-
ThermoRoute AI — adaptive on-device inference optimized for Arm-powered Android devices.
-
Real SCAM detection using Tiny INT8 with 95% confidence and no unnecessary model escalation.
-
Live device-aware routing with model latency, backend, telemetry, and decision trace.
-
Real SAFE classification with 100% confidence, processed fully offline on-device.
-
Physical Arm64 benchmark: Tiny INT8 is 3.69× faster and 93.5% smaller than Large FP16.
Inspiration
What it does
How we built it
Inspiration
Mobile AI apps often run the same model for every input, even when a smaller and faster model is already confident enough. This wastes compute, increases latency, and ignores the phone’s current battery, memory, and thermal condition.
ThermoRoute AI explores a better approach: start small, measure confidence, inspect live device conditions, and escalate only when the larger model is genuinely needed.
What it does
ThermoRoute AI is a fully offline Android application that performs adaptive SMS scam classification on Arm-powered devices.
Every message first runs through a Tiny INT8 model. The routing policy then considers:
- model confidence
- battery level
- charging state
- power-saving mode
- thermal status
- available memory
- latency budget
When confidence is sufficient, the Tiny INT8 result is returned immediately. When confidence is low and the device can afford additional compute, ThermoRoute escalates to a larger FP16 model.
The app displays the selected model, prediction, confidence, backend, latency, device telemetry, and a deterministic decision trace.
How we built it
The Android application was built with Kotlin, Jetpack Compose, TensorFlow Lite, and XNNPACK.
The ML pipeline uses the UCI SMS Spam Collection. Exact duplicates were removed before a deterministic train, validation, and test split. Two TensorFlow Lite models were exported:
- Tiny INT8 — efficient first-stage model
- Large FP16 — confidence-triggered fallback model
Both models, the vocabulary, evaluation subset, and metadata are packaged inside the APK. No cloud API, server, login, or internet connection is required.
Arm optimization
ThermoRoute combines multiple optimization techniques rather than relying on model compression alone:
- full-integer INT8 quantization
- FP16 fallback model
- XNNPACK CPU execution
- confidence-based model cascading
- live device-aware routing
- memory-mapped model assets
- single-threaded inference
- separate inference and end-to-end latency reporting
Physical Arm64 results
The benchmark was executed on a physical Samsung SM-A156E running Android 16 with the arm64-v8a ABI. Both models used XNNPACK on the CPU with one thread.
| Metric | Tiny INT8 | Large FP16 |
|---|---|---|
| Average inference latency | 0.05 ms | 0.17 ms |
| P50 latency | 0.03 ms | 0.15 ms |
| P95 latency | 0.11 ms | 0.34 ms |
| Accuracy on-device | 98.5% | 98.0% |
| Model size | 33.9 KB | 519.3 KB |
| App Java heap | 11.57 MB | 13.65 MB |
Measured comparison:
- 3.69× faster inference
- 93.5% smaller model
- 98.5% measured Tiny INT8 accuracy
- +0.50 percentage-point accuracy difference for Tiny INT8
The benchmark used 20 warmup runs and 200 measured runs. These numbers represent model inference latency, not preprocessing or complete end-to-end latency.
Battery and thermal savings are deliberately not claimed because a single benchmark run cannot establish them reliably.
Challenges
The largest challenge was keeping every result technically honest across different environments. Emulator execution was useful for Android correctness testing, but it was kept separate from physical Arm performance evidence.
Another challenge was supporting INT8 and FP16 execution through one Android inference pipeline while preserving reproducibility, clear timing boundaries, and an understandable routing trace.
Accomplishments
- Built a functioning adaptive inference system instead of a static benchmark
- Validated both models on a physical Arm64 Android phone
- Achieved a 93.5% model-size reduction
- Achieved a 3.69× measured inference speedup
- Preserved 98.5% measured on-device accuracy
- Implemented fully offline and private inference
- Added reproducible training, conversion, evaluation, and Android build workflows
- Exposed every routing decision directly in the user interface
What we learned
Model optimization is not only about making one model smaller. Mobile efficiency also depends on deciding when additional computation is justified.
We also learned that preprocessing can dominate total response time for very small neural networks. For that reason, ThermoRoute reports inference, preprocessing, and end-to-end latency separately.
What’s next
- validate performance across more Arm-powered phones
- conduct controlled thermal-soak and battery-drain studies
- extend the routing system to other mobile AI workloads
- improve preprocessing performance
- evaluate newer text representations
- compare additional Arm-compatible runtimes
Privacy and limitations
All inference runs locally. The application has no server, account, analytics, or external message transmission.
The current evaluation focuses on English SMS scam classification using one primary dataset and one physical Arm64 device. Results may vary across hardware, Android versions, and workloads.
Log in or sign up for Devpost to join the conversation.