Arm Offline AI Tutor

An on‑device Arm AI tutor for math and basic ML that runs a tiny transformer entirely on your phone, delivering fast, private explanations and quick checks with zero network.

Comment

Inspiration

  • Most AI tutors today depend on the cloud, which means latency, connectivity issues, and sending student data to remote servers. For STEM learners, especially in emerging markets, this is not always acceptable or even possible.
  • I wanted to see how far we can push on‑device AI on Arm phones: can we run a tiny transformer tutor fully offline, with predictable latency, on everyday Android devices?

What it does

Arm Offline AI Tutor is a mobile app that:

  • Runs a tiny transformer model entirely on an Arm‑based Android phone (no network needed).
  • Lets students chat in natural language about math and basic ML concepts (limits, derivatives, gradients, loss, etc.).
  • Generates step‑by‑step explanations plus quick‑check questions to reinforce understanding.
  • Displays real‑time on‑device latency metrics (load time and average inference time) so users and judges can see how well it performs on Arm.

How we built it

  • Designed a compact transformer model (TinyTutorModel) in PyTorch.
  • Exported it to TorchScript Lite (.ptl), optimized with optimize_for_mobile.
  • Integrated PyTorch Mobile Lite (pytorch_android_lite) into a native Kotlin Android app.
  • Loaded the Lite model from assets/, encoded user text into token IDs, and ran inference on‑device.
  • Built a simple chat UI with message bubbles and a lightweight rule‑based layer on top of the model outputs to structure explanations for learners.
  • Measured model load and inference time on device using System.nanoTime() and surfaced those numbers directly in the UI.

Challenges we ran into

  • Tooling changes: the original plan used ExecuTorch, but version and dependency conflicts forced a pivot to PyTorch Mobile Lite mid‑way.
  • Android build issues: Gradle plugin resolution, JVM heap limits, and native library loading (libpytorch_jni.so vs Lite runtime) all caused crashes that had to be debugged and fixed.
  • Handling large model artifacts in Git and Android assets while keeping the project manageable for submission.
  • Balancing model size vs. latency so the tutor feels responsive on real Arm devices, not just on a desktop emulator.

Accomplishments that we're proud of

  • A fully working offline AI tutor running a transformer model directly on an Arm phone, with no server component.
  • Clean, minimal Android app that clearly demonstrates on‑device inference performance (shape + latency) in the UI.
  • A reusable pipeline: Python export → TorchScript Lite → Android integration, which can later host stronger competition‑grade math tutor models.
  • Robust enough to run on both higher‑end (Pixel 8a) and more modest Arm devices.

What we learned

  • On‑device AI on Arm is very feasible when you control model size, sequence length, and runtime carefully.
  • The choice of runtime (ExecuTorch vs PyTorch Lite) and binary compatibility on Android can dramatically impact developer velocity.
  • Even a small transformer, combined with good instructional design (step‑by‑step explanations + quick checks), can already create a useful tutoring experience.
  • Clear latency measurements help communicate the value of on‑device AI to both users and judges.

What's next for Arm Offline AI Tutor

  • Plug in a stronger math/Olympiad tutor model (trained separately, e.g., via a Kaggle project) using the more advanced LLM Model + Android pipeline.
  • Expand the curriculum beyond calculus and basic ML to deeper problem‑solving patterns and quizzes.
  • Add features like progress tracking, spaced repetition, and offline problem sets.
  • Polish the UI/UX and publish a Play Store release under the production application ID kts.ai.arm.off.tutor.

Built With

Share this project:

Updates