A64Forge

Inspiration; AI agents often use one large model for every task, wasting memory and CPU. We built A64Forge to optimize each workflow stage independently for real Arm64 hardware.

What it does; A64Forge benchmarks GGUF models and llama.cpp configurations for classification, extraction, tool selection, reasoning, and summarization. It protects a quality threshold, finds Pareto-efficient candidates, and compiles the result into stage-aware routing and verifiable evidence.

How we built it; We used Python, FastAPI, Typer, SQLite, React, TypeScript, and llama.cpp. The system records latency, memory, throughput, quality, hardware capabilities, model hashes, and configuration details. It generates JSON, YAML, Docker, and HTML artifacts. A native aarch64 GitHub runner measured 20 configurations. Compared with the all-Qwen3-4B baseline, A64Forge achieved: 67.8% lower average stage latency 42.9% lower average peak memory 244.5% higher average throughput 6.7% higher deterministic quality

Challenges and lessons; Our first smaller baseline failed classification quality. Instead of weakening the quality gate, we added a measured 4B candidate and made failure an explicit non-deployable result. We also had no paid Arm64 server, so we created a bounded GitHub Actions workflow for genuine native measurements. We learned that optimization should happen per workflow stage: smaller models work well for some tasks, while others still require a larger model to preserve quality.

whats next for A64Forge; We plan to add more agent-workflow adapters, remote Arm64 workers, additional inference runtimes, and automated performance-regression testing.