SiliconScribe

What it does

SiliconScribe is a hardware design agent that runs entirely on an ARM laptop. A small language model reads a plain-English spec, writes Verilog, then checks its own work by running Icarus Verilog and Yosys. If the simulation fails, it reads the error and tries again. No cloud, no API keys.

We used this agent as a testbed to answer a practical question: how do you actually make a local LLM run faster on an Arm CPU?

The optimization

Arm publishes KleidiAI, a library of hand-written routines for the matrix math that LLMs spend most of their time on. We found that the quantization format decides whether KleidiAI runs at all. It has kernels for Q4_0 and Q8_0, but not for Q4_K_M, which is the file most people download by default. Use Q4_K_M and the Arm-optimized code never runs, with no warning.

Changing only the file format, same machine, same build:

Faster on every axis and smaller on disk. No trade-off.

The honest null result

We also tested simply turning KleidiAI on without changing the format. On this M2, that made no measurable difference: +2.1% reading, -6.2% writing, both inside normal run-to-run variation. We measured it twice, at the engine level and through the agent, and both agreed. The lesson is not "turn KleidiAI on," it is "give KleidiAI a format it can use."

How we made sure KleidiAI was actually running

The official llama.cpp binaries for Apple ship with KleidiAI disabled. Setting the build flag is not proof it works, so we verified twice: we inspected the compiled library for the Arm code (nm ... | grep kleidi returns 37 symbols vs 0), and we watched the program's own load-time log confirm the kernels were active on every trial. We also had to compile out Metal and Apple's Accelerate BLAS, because otherwise those libraries do the math instead and mask any effect.

What we learned

The biggest lesson was measurement discipline. An early run showed a 79% speedup that turned out to be thermal contamination from a previous test. Every number in the repo is now from a cold machine with cooldowns between runs, and we kept a record of the bad measurement as a warning. The agent's own self-checking testbenches also repeatedly caught bad information from other sources during the build.

Challenges

Isolating the Arm CPU path from Metal and BLAS, a runaway-generation bug where the model resubmitted identical broken code, and separating a real signal from thermal noise on a passively-cooled 8GB laptop.

Full methodology, per-figure config, and reproduction steps are in the repo README.

Built With

• ai
• apple-silicon
• arm
• edge
• icarus-verilog
• kleidiai
• llama.cpp
• python
• quantization
• qwen
• rtl
• verilog
• yosys