Project Entity

Give me an Arm phone and a GGUF. ENTITY reads the silicon, tunes itself to it and proves which decision earned the speedup.

Comment

posted an update —

[1.6.0]

The biggest release since 1.0.0, on three fronts at once: the runtime got faster where it was weakest, the app became a real daily tool, and the UI got the polish pass it deserved. On the inference side, prompt processing now runs on all eight cores through a dedicated thread pool while generation stays pinned to the four Cortex-A78s — prompt-heavy turns start noticeably sooner, and decode keeps its bandwidth-bound sweet spot. A conversation that outgrows the context window now trims its oldest turns (system prompt preserved) and keeps going instead of failing, and two latent position-accounting bugs in that path were fixed along the way. On the app side, chats finally persist: every conversation is stored in a local SQLite database, survives process death, and can be switched, renamed, or deleted from a new Conversations menu — with a new engine API that re-primes the KV cache from stored history so a restored chat continues seamlessly. Rotation no longer loses state (a proper ViewModel owns the chat now), generation survives backgrounding, the system prompt is editable in Settings, and assistant answers render markdown with long-press Copy/Regenerate. The UI was refreshed end to end — softer bubbles, a typing indicator, subtle entry animations behind a new Animations toggle that also honors Android's Remove-animations accessibility setting. The in-app benchmark grew up too: multi-run with median ± stddev, thermal-gated cooldowns between passes, TTFT, and CSV export. And the release build finally acts like one: R8-minified with stripped native symbols.

---> APK drop from ~100 MB to ~7 MB.

Added

  • Chat persistence + multiple conversations — chats stored in a local SQLite DB (chats.db), auto-titled from the first message; a Conversations menu lists them (tap to switch, long-press to rename/delete); the most recent conversation is restored on launch, and partial answers are saved if you hit Stop.
  • primeHistory engine API — rebuilds the KV cache from a stored conversation without generating, so a restored or switched chat continues exactly where it left off.
  • System-prompt editor (Settings) — multiline editor with reset-to-default; used on every load/new-chat/re-prime.
  • Markdown rendering — bold, italic, inline code, fenced code blocks, bullets, and headings in assistant messages; hand-rolled renderer, parsed once per completed message and cached.
  • Long-press message actions — Copy any message; Regenerate the last answer.
  • Animations toggle (Settings, default on) — one switch disables all app animations instantly, and Android's "Remove animations" accessibility setting is honored automatically.
  • Multi-run benchmark — 1/3/5 runs per configuration with median ± stddev, thermal-gated cooldown between passes (live temperature status), derived TTFT, and CSV export via SAF.
  • Release build — R8-minified, resource-shrunk, native symbols stripped: ~7 MB APK vs the ~100 MB debug build; signed for sideloading.

Changed

  • Prompt processing uses all 8 cores — a dedicated ggml thread pool spans the whole SoC during prompt evaluation while generation keeps its own pool pinned to the 4 big cores (llama_set_n_threads per phase, thread-pool functions resolved at runtime for the GGML_BACKEND_DL build, with a clean fallback). Faster time-to-first-token on long prompts; decode unchanged by design.
  • UI polish pass — refined bubbles (softer radii, subtle strokes), typing-dots indicator while generating, message entry fade-rise, ripple + press-scale on the send button, refreshed input bar; both themes, no new libraries, no bitmaps, hardware-accelerated view-property animations only.
  • Rotation/theme changes keep everything — chat and in-flight generation now live in a ViewModel; the engine is only torn down when the app actually exits, and generation continues while backgrounded.

Fixed

  • Context-full no longer breaks a long conversation — the KV window trims its oldest turns (system prompt preserved) with correct position accounting; previously a mid-generation or mid-prompt shift could desync positions and corrupt the remaining budget.
  • JNI teardown hardening — engine error paths free their batch/context, sampler updates are create-then-swap, unload is double-free safe, and destroying the engine from an error state no longer crashes.
  • Per-token allocation removed on the partial-UTF-8 streaming path.

Log in or sign up for Devpost to join the conversation.