optimus-mobile-ai

• 

  Benchmark summary showing the optimized on-device int8 model, model-size reduction, accuracy parity, and local-only privacy mode.

• 

  CLI demo showing offline mobile AI prediction and benchmark output generated locally without a cloud inference call.

• 

  On-device inference flow: user input is processed locally, optimized with int8 quantization, and returned as a private mobile action hint.

Inspiration

What it does## Inspiration

Mobile AI is powerful because it keeps useful intelligence directly on the device in the user’s hands. Phones, tablets, and Arm-powered laptops should be able to classify, summarize, translate, or assist with text without always depending on a cloud endpoint. Optimus-mobile-ai was inspired by the Arm Create: AI Optimization Challenge goal of making on-device AI smaller, faster, private, offline-capable, and easier to validate on Arm client devices.

What it does

Optimus-mobile-ai is a Track 3 Mobile AI project that simulates an offline on-device assistant for short user messages. It classifies each message into an intent such as reply, schedule, summarize, or translate.

For each message, it also returns urgency, a local action hint, and a privacy mode showing that inference is local-only. The project compares a baseline fp32 inference path with an optimized int8 quantized path and reports model size, latency, throughput, accuracy, CPU architecture, Python version, NumPy version, and estimated battery budget.

How we built it

We built Optimus-mobile-ai as a standalone Python project using NumPy. The model uses deterministic hashed text features and a compact intent classifier trained with ridge regression.

The optimized inference path quantizes fp32 model weights into int8, quantizes message vectors, performs integer dot-product inference, and dequantizes only the final intent scores. We also added a CLI, environment reporter, benchmark exporter, local-only privacy metadata, battery-aware benchmark estimates, and unit tests.

Challenges we ran into

The main challenge was designing a Mobile AI workload that clearly shows on-device value without requiring a full Android or iOS app. We focused on the core mobile AI behavior: local inference, privacy, offline use, responsiveness, and reduced model size.

Another challenge was preserving prediction quality after quantization. The int8 path needed to reduce model size while still matching the fp32 baseline on the sample evaluation set.

Accomplishments that we're proud of

We are proud that Optimus-mobile-ai creates a clear Mobile AI optimization story: local-only inference, fp32 baseline, int8 optimization, benchmark output, and reproducible validation.

The current benchmark shows a 73.71% model size reduction while maintaining accuracy parity between fp32 and int8 inference. We are also proud that the project is standalone, easy to run, and designed around real mobile constraints such as privacy, responsiveness, offline use, and battery awareness.

What we learned

We learned that Mobile AI optimization is about more than model speed. A useful on-device AI project should also consider privacy, memory footprint, offline capability, latency, and battery impact.

We also learned that int8 quantization is a practical starting point for Arm-powered client devices because it reduces model size and memory use while keeping inference local.

What's next for optimus-mobile-ai

Next, we want to run the benchmark on real Arm-powered client devices such as Android phones, Android tablets, and Windows on Arm laptops.

Future versions could add ONNX Runtime, ExecuTorch, LiteRT or TensorFlow Lite support, a small Android demo app, local language packs for translation, battery profiling, and a mobile UI for private offline message assistance.

Built With

• ai
• android
• arm64
• benchmarking
• int8
• machine-learning
• mobile-ai
• model-optimization
• numpy
• offline-ai
• on-device-ai
• privacy
• python
• quantization
• windows-on-arm