REVIVE

About five billion smartphones are alive right now and doing nothing. Not broken, not obsolete, just sitting in a drawer because something newer came out. Every one of them has a processor fast enough to run real neural network inference, a neural engine that Apple or Qualcomm spent billions designing, and enough RAM to hold a meaningful chunk of a language model. Most of them will eventually end up in a landfill.

We thought that was a waste worth fixing.

REVIVE runs a single language model across a cluster of whatever devices you have lying around. A phone, a laptop, a Raspberry Pi, another phone. The model gets split into consecutive slices, one per device. Each device holds only its chunk. When the system generates a word, the data flows through every device in order, one lap around the ring, and the last device in the chain produces the output. Unplug a device and generation stops. Plug it back in, the cluster recovers. The output is numerically identical to running the whole model on one machine.

The model itself is a fine-tuned version of Qwen3, a small open-weight language model from Alibaba. We trained it using knowledge distillation: we ran a much larger Qwen3 model on thousands of examples and used its outputs as training targets, so the smaller model learns to approximate the reasoning of something ten times its size. The result is a model that fits on consumer hardware but behaves sharper than its parameter count suggests.

How the ring actually works

Each device runs a small HTTP server. On startup, it loads its assigned layers of the model into memory, nothing else. It registers with the cluster controller, reports its layer range, and waits. When a query comes in, the first device in the ring tokenizes it and runs a forward pass through its layers. The output is a hidden state tensor, a floating-point matrix that encodes everything the model has "thought" so far. That tensor gets serialized and sent over the network to the next device, which runs its layers on top of it and forwards the result again. The last device in the chain runs its layers, samples a token from the output distribution, and sends that token back to the start of the ring. Then it all happens again for the next token.

The hidden states are big. For Qwen3-4B with a sequence length of 512, each transfer is around 2 MB. At 15 tokens per second across three devices, that's about 30 MB/s of sustained network traffic. A normal home WiFi connection handles 60 MB/s without trying. The bottleneck is the phones doing the matrix math, not the link between them. We were surprised by that too.

The iOS app uses llama.cpp under the hood, compiled with Metal support so the inference runs on the Neural Engine. Each device is configured with a layer start and layer end, and the llama.cpp context is initialized with those bounds so it genuinely only runs that slice of the network, not the full model.

The Arduino

There is a $10 Arduino Uno sitting between the laptop and the rest of the cluster, plugged in over USB. It is the cluster's brain. Not for inference. For control.

The Arduino runs our BMC firmware, about 5KB of C. It owns the partition table: which device covers which layers, who is first in the ring, who is last. Every device sends a heartbeat over the network every three seconds. The Arduino watches those heartbeats. If a device misses two in a row, the Arduino marks it dead, recalculates the partition to cover the gap, and broadcasts the new assignment to the survivors. Unplug the Arduino and the cluster has no authority. It stops taking new queries because nobody is in charge of deciding who does what.

The 2KB RAM constraint is real and it bit us. We initially wanted dynamic routing tables, per-device retry queues, sequence tracking across multiple in-flight requests. None of that fits. What fits is a flat array of six worker structs, a state machine with three states, and a line-oriented ASCII protocol over serial. Turns out that's enough. The simplicity makes it reliable in a way that more complex firmware often isn't.

We also built a software stand-in for the Arduino that runs on the laptop and speaks the exact same protocol. That let us develop and debug the entire cluster, including the failure and recovery scenarios, without touching any hardware. Keeping the software simulator and the real firmware in sync on a shared protocol spec was its own discipline but paid off every time we needed to test something fast.

The fine-tuned model

Running Qwen3-0.6B raw is fine. Running a version of it that was trained against Qwen3-4B is noticeably better on the kinds of tasks we care about. We generated a distillation dataset by running the larger model on a few thousand examples across eight different roles: reasoning, drafting, summarizing, critiquing, fact-checking, and a few others. The smaller model was then fine-tuned on those outputs using QLoRA, which lets you train a model with a fraction of the memory a full fine-tune would need.

We ran that training on AWS SageMaker with the job partitioned across the 8 roles in sequence, and pulled the checkpoints back down when it finished. The whole training run cost about a dollar and took 35 minutes.

The refinement layer on top of the cluster output goes one step further. After the on-device swarm finishes generating, we take its raw chain of thought and pass it to Claude Sonnet via Bedrock. Sonnet reads what the small models were trying to say and writes a polished final answer grounded in that reasoning. The thinking happens on your devices. The polish happens in the cloud. You get the privacy properties of local inference and the output quality of something much larger.

What we got working

The cluster produces the same tokens as the single-machine reference for the same inputs. That correctness check was the first hard thing we had to get right, and we spent most of a debugging session on it before the outputs converged.

The Arduino genuinely runs the cluster. Not simulated. The physical board plugged into USB, running the C firmware, sending partition updates over serial. You can open a serial monitor and watch the heartbeat protocol in real time.

The kill and heal demo works. Unplug a phone mid-generation. The Arduino spots the dead heartbeat, repartitions, and the cluster halts cleanly rather than wedging. Plug the phone back in, it re-registers, the Arduino broadcasts the restored partition, and new queries flow again.

And the fan on the Arduino's PWM output ramps up when inference starts and ramps back down when it finishes. It sounds like a small thing. It is a small thing. But there is something satisfying about a $4 piece of hardware responding to inference load in real time.

What we learned

Pipeline parallelism across home WiFi is genuinely viable. The folklore that it is too slow turned out to be wrong, at least at the model sizes we care about. The compute is the constraint, not the network.

A microcontroller is actually a good place for a cluster's control plane. It cannot crash the same way a software process can. It does not share a heap with the workers. If the laptop running the coordinator dies, the Arduino still knows who is alive and can keep reporting state. There is something to be said for putting the brain on dedicated hardware with a dedicated power rail.

Small models trained carefully against large models are genuinely useful. We did not need a 70B parameter model to get good answers. We needed a 4B model that learned from one.