Pi Arm AI Optimizer: Running Production AI Agents on ARM64 with Ollama

Submitted to the Arm Create: AI Optimization Challenge — Physical AI track (edge devices, embedded, autonomy)

Project Overview

This project demonstrates real-world AI optimization on ARM-powered platforms by running a production AI agent entirely on a Raspberry Pi 5 (ARM64, 8GB RAM). The agent manages email, writes and deploys code, monitors services, and submits content — all using locally-quantized LLMs via Ollama.

The optimization work focuses on making small (3B and below) language models viable for agentic workloads on ARM64 hardware with severe memory constraints. This is the most constrained ARM platform that can still run useful AI workloads — a $80 Raspberry Pi 5. If you can make a 3B model do production agent work on a Pi 5, those same optimizations apply to every ARM device in the ecosystem.

Why This Should Win

• Real production usage: This isn't a benchmark demo — it's a live AI agent that has been running 24/7 for 3+ months, performing real tasks (email management, code deployment, content submission, service monitoring)
• Measurable optimizations: Detailed benchmarks across 5 models with real tokens/sec, RAM usage, thermal metrics, and task success rates
• Reusable patterns: The task decomposition and context management strategies are applicable to any ARM edge AI deployment
• Extreme constraint optimization: Optimizing for 8GB RAM total (including OS) is harder than optimizing for cloud instances with 64GB+

Built With

• hermes
• ollama
• raspberry-pi