Zero-Assist

Zero-Assist: Privacy‑first on‑device AI assistant for Android. Runs fully offline(via local models if any available), automates apps, controls IoT, and speaks naturally – no cloud required.

Comment

Inspiration

Most AI assistants send your conversations to the cloud. You lose privacy. They stop working offline. I wanted an assistant that respects you – one that lives entirely on your phone, understands natural language, controls your apps, and even talks to your smart home. Zero-Assist is that assistant.

What it does

Zero-Assist is a privacy‑first, on‑device AI assistant for Android. It:

  • Runs 100% offline using local LLMs (or optionally uses cloud models)
  • Automates other apps (Spotify, WhatsApp, settings) via Android AccessibilityService
  • Controls IoT devices through built‑in MQTT – lights, sensors, Arduino, ESP32
  • Speaks naturally with local neural TTS (Piper ONNX) – no cloud dependency
  • Executes code in a secure sandbox (Termux + PRoot) – analyse data, generate charts
  • Runs a full Linux environment (Alpine + XFCE) on‑demand for advanced users
  • Provides a visual workflow editor – drag‑drop automations, no coding required

You can say "Turn on the living room light", "Send 'I'll be late' to Mom on WhatsApp", or "Analyse this CSV and email me a chart" – Zero-Assist does it all without sending a byte to the cloud.

How I built it

Zero-Assist is a hybrid Android + Rust application.

  • Frontend: Kotlin + Jetpack Compose
  • AI Core: Rust (ZeroClaw) – tool calling, memory, cron, event‑driven SOPs – exposed to Kotlin via UniFFI
  • Workflow Engine: Embedded Flow‑Like (Rust) – local visual automation
  • App Automation: Android AccessibilityService with custom overlay and gesture simulation
  • IoT & MQTT: Native MQTT client – subscribes, publishes, reacts to sensor data
  • TTS: Piper ONNX models – offline, fast, natural voice
  • Code Execution: Termux + PRoot – run Python, shell, any Linux command
  • Linux Environment: Alpine Linux with XFCE, streamed via Termux‑X11
  • Storage: Room + SQLCipher (encrypted), plus shared folder for user files
  • Background Tasks: WorkManager for cron‑like scheduling

All communication between Kotlin and Rust happens through UniFFI – native performance, type safety, zero overhead.

Challenges I ran into

  • AccessibilityService reliability – Modern Android kills background services aggressively. Solved with foreground service + persistent notification and user education.

  • Rust‑Kotlin bridging – Managing UniFFI bindings for a growing toolset required careful .udl design and versioning. Each new tool needed updates in three places (Rust, UDL, Kotlin).

  • Background shell execution – Runtime.exec() is heavily restricted on newer Android. Integrated Termux to run commands without root.

  • 32‑bit device support – Flow‑Like’s Lance dependency has no 32‑bit SIMD fallback. Decided to support only 64‑bit devices for the workflow engine, keeping core assistant functional on 32‑bit.

  • TTS latency – Piper needed ONNX optimisation and model tuning, but the result is near‑instant offline speech.

Accomplishments I'm proud of

  • Fully functional offline AI – Works on a plane, in a tunnel, anywhere.
  • Seamless cross‑app automation – "Play Taylor Swift on Spotify" opens the app, searches, and plays without manual intervention.
  • Real‑time IoT control – MQTT integration lets Zero-Assist react to temperature, motion, and water leak sensors instantly.
  • User‑controllable voice – Users can bring their own free offline Piper ONNX voice – no cloud, no cost, full privacy.
  • Clean, modular architecture – Rust core + Kotlin UI + UniFFI makes the codebase maintainable and fast.
  • Visual workflow editor – Non‑technical users can now create automations by dragging nodes.
  • Linux VM inside the app – A full Alpine desktop runs on‑demand, giving Zero-Assist the power of a real OS.

What I learned

  • Building a local‑first AI assistant is not just possible – it's the future. Users deeply care about privacy and offline capability.
  • Rust + Kotlin via UniFFI is a production‑ready stack for Android. It eliminated crashes and improved performance significantly.
  • AccessibilityService is powerful but fragile – you must design for failure (fallbacks, retries, user guidance).
  • Offline TTS quality has come a long way; Piper delivers excellent, low‑latency speech without any cloud dependency.
  • MQTT is the unsung hero of home automation – lightweight, event‑driven, and easy to integrate.
  • A hackathon submission is more than code – storytelling matters as much as features.

What's next for Zero-Assist

  • On‑device memory with RAG – Store facts and conversations in a local vector database (sqlite‑vec)
  • Voice cloning – Let users clone their own voice or use celebrity voices (Pocket‑TTS / NekoSpeak)
  • Self‑healing tool routing – Automatically recover from API failures without re‑invoking the LLM
  • Plugin store – Allow third‑party WASM plugins (sandboxed) for community‑built actions
  • Multi‑device sync – Securely share settings, memory, and workflows across phone, tablet, and desktop
  • Open‑source the core – Release ZeroClaw (Rust engine) under MIT license for others to embed
  • Google Assistant integration – Replace the default assistant with Zero-Assist (using VoiceInteractionService)

We believe the future of AI is private, local, and user‑owned. Zero-Assist is one step in that direction.

Built With

  • android
  • flow-like
  • jetpack-compose
  • kotlin
  • mqtt
  • piper-tts-(onnx)
  • rust
  • termux
  • uniffi
Share this project:

Updates