Rotondo Farms NPU

A real grow-cabinet monitors soil moisture and narrates plant health using a local LLM on the Orange Pi 5's NPU — with a measured CPU vs. NPU benchmark showing exactly where NPU inference pays off.

Comment

Inspiration

I've been building Rotondo Farms — an automated indoor grow cabinet — for a while, with soil moisture sensors feeding a dashboard. It always just showed raw numbers. I wanted the system to actually tell me something about the plants in plain language, and I wanted to do it fully on-device, no cloud API calls. The Arm Create challenge was the perfect excuse to actually build that, and to answer a question I was genuinely curious about: does running a small LLM on the Orange Pi 5's NPU actually beat running it on CPU, for a real, small, real-time workload — or is that assumption wrong?

What it does

Rotondo Farms reads live soil moisture from capacitive sensors (currently monitoring a tomato and a pepper plant) via an ADS1115 ADC over I2C, straight into an Orange Pi 5. Those readings get built into a prompt and sent to a locally-hosted Qwen3-0.6B model — either running on CPU (GGUF format) or natively on the RK3588S NPU (.rkllm format) — which generates a short, natural-language summary of plant health, calling out anything that needs attention.

The project also includes a --trials benchmark mode that runs controlled, repeatable, back-to-back comparisons between the CPU and NPU paths on identical input, logging every result to CSV.

Benchmark result (8 trials each):

  • CPU (GGUF): 39.15 tok/s avg (range 25.9–41.7)
  • NPU (native .rkllm, W8A8): 13.43 tok/s avg (range 12.4–14.4)

CPU actually won, and I think that's the more interesting result than if NPU had simply "won" as expected — see below.

How we built it

  • Hardware: Orange Pi 5 (RK3588S, 6 TOPS NPU), ADS1115 ADC, capacitive soil moisture sensors wired over I2C.
  • Inference server: RKLLama, hosting Qwen3-0.6B for both a CPU (GGUF) path and a native NPU (.rkllm) path.
  • Pipeline: A Python script reads the sensors, builds a prompt, calls the local inference server's chat API, and logs timing/token data for every call.
  • Benchmark methodology: A fixed prompt (not live sensor data, to eliminate drift between runs) was run 8 times against each backend, and results were averaged.

Challenges we ran into

Getting the model to genuinely run on the NPU — not silently fall back to CPU — was most of the real engineering work here, not the sensor pipeline itself:

  1. GGUF-on-NPU never actually engaged the NPU. The first "working" NPU setup looked fine but was confirmed, via /sys/kernel/debug/rknpu/load staying at 0% during generation, to be running entirely on CPU the whole time. GGUF-on-NPU support in RKLLama is explicitly marked experimental — for good reason.
  2. Switching to the natively-supported .rkllm format got real NPU utilization (confirmed 3–50% load) — but generation started truncating mid-sentence, consistently, across three different model files and sizes.
  3. Root cause: an ABI mismatch. The RKLLM runtime library had been updated with a different C struct layout, but a known-good older runtime version was required for stable generation on this hardware. Pairing the old runtime .so with newer application code (mismatched struct field offsets) silently corrupted a parameter passed into the native library. Fixed by pinning both the runtime library and the application code to the last mutually-compatible commit — a real, traceable bug, not a config typo.
  4. Also diagnosed and fixed along the way: a background systemd service silently competing for the same port as manually-started test instances, and a pipeline bug where the model wasn't being explicitly reloaded between requests.

What we learned

The most valuable finding wasn't "NPU is faster" — it's that NPU acceleration isn't automatically the right choice for every model size. At 0.6B parameters, the model is small enough that the NPU's per-request dispatch and initialization overhead outweighs its parallel-processing advantage over a full generation. NPU acceleration is expected to pay off more on larger models or batched workloads, not single small real-time requests like this one. That's a genuinely useful thing to know before reaching for NPU offload — knowing where the crossover point sits matters as much as knowing the hardware exists.

What's next

Closing the loop on the physical system — I already have a self-watering subsystem (relay, float switch, pump) wired and ready, just not yet integrated with this NPU pipeline. The longer-term plan is a fully closed-loop system: sensors → NPU narration → automated watering decisions, still entirely on-device.

Built With

Share this project:

Updates

Submission history