Project Entity

Give me an Arm phone and a GGUF. ENTITY reads the silicon, tunes itself to it and proves which decision earned the speedup.

Comment

posted an update —

[2.0.0]

The headline: universal Arm support via runtime CPU backend dispatch. ENTITY previously shipped a single CPU backend compiled for one known SoC. v2.0.0 ships 7 Arm CPU backend variants (armv8.0, armv8.2×2, armv8.6, armv9.0, armv9.2×2, each with KleidiAI kernels) and dynamically loads the best one the phone's CPU supports at startup — so the app now runs on essentially any Arm Android phone without crashing on older cores or missing faster kernels on new ones. A new first-run "Optimize for your device" dialog detects the loaded backend's ISA features (e.g. i8mm, dotprod, SVE) and suggests Auto mode. The big-core affinity optimization is unchanged and was already SoC-agnostic.

A power measurement bug was fixed: many OEM kernels report BATTERY_PROPERTY_CURRENT_NOW in milliamps instead of the documented microamps, causing 1000× underreporting on affected devices; a new PowerMath helper uses physical plausibility instead of trusting the unit. This fix is confirmed on hardware via cross-device validation.

Cross-vendor measurement: the core optimization was independently validated on a second device (Qualcomm Snapdragon 6 Gen 4) using the same model and protocol as the reference device. Reference device (MediaTek Dimensity 7300): 8.0 → 17.7 tok/s (+121%), 1.7 → 4.2 tok/W (2.5×). Second device (Snapdragon 6 Gen 4): 6.0 → 13.1 tok/s (+117%), 1.8 → 3.8 tok/W (2.1×). The core optimization reproduces across vendors without vendor-specific code.

The in-app benchmark now measures ENTITY's shipped Auto configuration (split thread pools for prompt/decode) instead of an explicit 4-thread test, so the result accurately reflects what users get. UI strings are now SoC-neutral and report detected core counts and ISA features at runtime instead of hardcoding Cortex-A78 names. Eight new unit tests cover device detection and power-unit resolution.

Added

  • Runtime CPU backend dispatch — 7 Arm variants with automatic selection at startup via ggml_backend_load_all_from_path + scoring. No UI toggle, no config — just works.
  • First-run "Optimize for your device" dialog — detects perf/efficiency-core counts, ISA features (i8mm, dotprod, SVE2, etc.) from the loaded backend, suggests Auto mode. "Not now" to dismiss; can be re-run from Settings.
  • SoC-neutral model-info string — reports actually-detected core counts and ISA features, not hardcoded Cortex-A78 names; works on MediaTek, Qualcomm, Samsung, etc.
  • PowerMath helper — resolves BATTERY_PROPERTY_CURRENT_NOW unit (microamps vs milliamps) by physical plausibility instead of trusting the documented unit; fixes 1000× underreporting on affected OEM kernels (Qualcomm especially).
  • Unit tests — app/src/test/java/com/example/llama/DeviceOptimizerTest.kt: 8 JUnit4 tests covering backend feature detection, core ranking, and device optimization; plus 6 tests for PowerMathTest.kt covering unit resolution and boundary cases.

Changed

  • In-app benchmark measures the shipped configuration — now runs Auto mode with split thread pools (generation on big cores, prompt processing on all cores) instead of an explicit 4-thread config. The naive control (8 threads, all cores) unchanged; protocol (PP 512 / TG 128) unchanged.

Fixed

  • Power readout was 1000× wrong on some devices — OEM milliamp/microamp confusion now resolved via plausibility, not trust-the-unit. Affects only power/efficiency display, not tok/s measurements.
  • Power graph received garbage on unsupported devices — getIntProperty sentinel value was feeding bad data; now guarded.

Log in or sign up for Devpost to join the conversation.