Inspiration
Every year, millions of working computers are thrown away because they're not fast enough to keep up. We wanted to give them a second chance: running AI together. Good AI costs money, sends your data to someone else's server, and has environmental impact . Free open models like Ollama avoid both, but they're limited to the how good is your device.
Meanwhile, laptops and computers are all around us. Our three laptops sat in the same house all year around. Together they have more memory than just any one of them. So we asked: what if computers could team up to run one AI model together, as easily as joining a group chat?
What it does
LowLlama turns a few ordinary computers into a small, private AI cluster.
- One laptop is the Master. It holds the AI model and is where you chat.
- The others are Helpers. They never download the model. They lend their memory and CPU.
- Joining takes an invite code. The Master creates a code that expires in 15 minutes. A helper uses it once to join, and gets its own private credentials for everything after.
- The Master keeps track of everyone. Each helper sends an authenticated "still here" heartbeat every few seconds. If a helper goes quiet for 10 seconds, it's marked offline. If it comes back, it reconnects as the same machine, not a duplicate.
- The model is split across machines. Different layers of the model live on different computers and work through each answer together.
What pooling does and doesn't do: it does not make a single answer faster. An AI model writes one small piece of text at a time, and each piece passes through every layer in order, so the computers take turns. What pooling gives you is more total memory, so you can run a bigger, smarter model than any one of your machines could hold.
How we built it
- AI engine: llama.cpp, the open-source engine underneath tools like Ollama. Its RPC feature lets one computer send parts of a model to others. Every machine runs the same pinned release (b12110)
- Model: Works with open source models, (eg Ollama)
- Private network: Tailscale, built on WireGuard. Every machine gets a private address that works across different WiFi networks and hotspots, with no port forwarding.
- Coordinator (FastAPI): runs on the Master, with endpoints to create invites, join, register, send active status, reconnect, and list nodes.
- Node registry and invites: track every machine and its status, validate and expire invite codes, and issue per-machine credentials.
- Helper agent: a Python program on each helper. It reads the machine's specs with psutil, joins with the invite code, stores its credentials securely using Windows DPAPI, and sends authenticated heartbeats.
- One-command setup: PowerShell scripts download the exact llama.cpp build and model and verify every file with a SHA-256 checksum, so nobody has to compile C++.
- Tests: automated unit tests for the registry, invite codes, coordinator API, agent, credential storage, and RPC checks
What works
- The model runs locally on the Master and answers questions
- llama.cpp splits the model between two devices
- Coordinator: invite, join, register, heartbeat, reconnect, and node list, with per-machine credentials
- Online/offline detection after 10 seconds of silence, without duplicate entries
- Helper agent: hardware detection, invite-based enrollment, encrypted credential storage, authenticated heartbeats
- One-command, checksum-verified setup scripts
- Automated tests across the coordinator and agent
Challenges we ran into
- Ollama can't split a model across computers. Our first idea was "Ollama, but distributed." That isn't possible, so we went one level deeper and used llama.cpp directly.
- Proving the helper really helped. An answer coming back isn't proof a second machine did work. We had to check the telemetry data all the time(RAM usage, CPU usage)
- Campus networks. University WiFi often blocks devices from talking to each other. Tailscale gets around this, but setting up a shared private network
- Securing a component with no authentication. We had to build our own trust layer (expiring invites, per-machine credentials, network isolation) around a worker that accepts anyone who can reach it.
- Teamwork under pressure. At one point we had several conflicting plans with different role splits. We stopped, picked one, agreed on a shared API, and split ownership by file. Along the way we fought merge conflicts, IDE settings files, and Windows line endings.
Accomplishments that we're proud of
- A working backend with secure enrollment, heartbeats, and automated tests, built by three people in parallel against a shared API.
- The model runs, and llama.cpp splits it across devices.
- Setup that needs no compiling: one script per machine, with checksum verification.
- Being upfront about what pooling can and can't do.
What we learned
- How large language models actually run: layers, tokens, memory, and why more computers doesn't mean more speed.
- How to design and test a REST API with FastAPI, and how to add authentication to it.
- How mesh VPNs like Tailscale and WireGuard connect devices across networks.
- How to split a project across a team: agree on the interface first, own separate files, and integrate often.
What's next for LowLlama
- A dashboard to see every machine, start the model, and chat in one place.
- Automatically splitting the model based on each machine's free memory.
- Recovering when a helper drops out mid-session.
- Saving the registry to a database so it survives restarts.
- A fully self-hosted network (Headscale) with no outside service.
Team
- MD Rodel Rahman: coordinator server (FastAPI endpoints)
- Itmam Shahid: AI engine (llama.cpp runtime, setup scripts) and helper agent
- Shafin T. Rahman: node registry and invite system
How to run it
Full instructions are in the README.

