Inspiration

Good open models need 16–20 GB just for their weights. Most student laptops have 8–16 GB. Cloud AI isn't an option for private notes, unpublished work or exam prep under data rules. But a study group always has several laptops in the room. What if they could act as one computer?

What it does

PoolMind runs one large model split layer by layer across several laptops. Only one laptop stores the model file; the others lend their memory and GPU. Together they run Qwen3-30B (18.6 GB), which doesn't fit on any one of our laptops at default settings.

  • Quick answers come from each laptop's own small QVAC model (Llama 3.2 1B).
  • Deep answers come from the pool. Every answer is labelled with where it ran.
  • Live pool dashboard: a layer map coloured by laptop, read from the engine's own load log; a card per laptop (online/offline, Metal/CUDA/CPU, layers held, memory used); "Does it fit?" bars; real tokens per second and time to first token; and an Offline badge.
  • Resilience: if a laptop leaves, Deep falls back to the local model within seconds ("Pool unavailable — answered locally"). When it comes back, the pool reloads on its own.

No cloud, no server, no internet needed.

How we built it

  • QVAC SDK (@qvac/sdk) runs everything: the local model on every laptop and the pool itself. Workers run QVAC's GGML RPC server (startRpcServer). The coordinator finds them (discoverRpcServers) and calls loadModel with split-mode: layer and a tensor-split to place the layers across the laptops.
  • Pears stack: workers announce themselves on a private Hyperswarm topic derived from a shared team secret. There's an optional hypertele tunnel over Hyperswarm (end-to-end encrypted, keys derived from the secret), so nobody types IPs or keys.
  • Coordinator (Node.js): a separate, supervised process. It checks worker heartbeats, serves an OpenAI-compatible endpoint behind a bearer token, and restarts and re-splits when a laptop leaves or joins.
  • App (Node.js + Express, plain HTML/CSS/JS): streaming chat plus the dashboard, with hard timeouts and a local fallback.

Challenges we ran into

  • A worker vanishing mid-token crashes the native RPC client, so the coordinator runs as its own supervised process and the app never goes down with it.
  • llama.cpp's RPC server serves one client at a time. Probing it for liveness made healthy workers look dead, so we added a separate heartbeat port.
  • By default the RPC server also exposes a CPU device, and layers landing there crash the worker. Each worker now exposes exactly one device.
  • Showing the real layer split instead of our own arithmetic: we parse the engine's per-device buffer log lines during the load.

Accomplishments that we're proud of

  • A 30B-parameter model split across laptops, with every layer's location shown live.
  • Measured on one MacBook over loopback, with worker and host as separate processes: Qwen3-30B-A3B split 2:1 ran at 64–69 tok/s with a ~0.2 s first token. Worker loss is detected in about 5 s, and Deep comes back without restarting the app.
  • Every number on the dashboard is measured, not invented.

What we learned

Splitting a model fits a bigger model; it doesn't make one answer faster. Every token crosses the network, so the link between laptops matters more than anything else. A cable beats a phone hotspot by orders of magnitude in latency.

What's next for PoolMind

  • Testing at scale with more laptops and mixed hardware (Mac + Windows CUDA).
  • Smarter splits weighted by each laptop's measured speed.
  • Re-splitting without a full reload when someone joins or leaves.

Built With

Share this project:

Updates

Submission history