Oasis

Inspiration

Law firms, clinics, hospitals and school systems hold sensitive records they can't send to a cloud AI. A server big enough for a strong model costs thousands of dollars, but every office and campus already has computers with capable GPUs sitting idle. The goal: turn those machines into one private AI server that never touches the internet.

What it does

Oasis pools the GPU memory of ordinary computers into a single model server on a private, offline network.

Discovery: each machine runs an agent that reports its GPU and free memory.
Placement: the coordinator decides which machine holds which slice of the model, then starts llama.cpp on every machine.
Live join: a new machine joins and the pool re-splits the model without anyone restarting things by hand.
Shared state: SpacetimeDB holds the state of the whole pool, and the dashboard reads it live.

Measured on three MacBook Airs (one M3 16 GB, two M5 24 GB) on a gigabit switch, no internet:

Gemma 4 26B (25.0 GiB) runs at 12.5 tokens/s with 0.6 s to the first token, 9 of 9 requests OK. The largest single Air can give a model 17.8 GiB, so the model is 1.4x larger than any one machine.
Speculative decoding (n-gram) speeds up rewrite tasks 1.6-1.7x.
The network round trip between machines is 0.5-0.9 ms, and each token moves about 6-9 KB per link.

Why SpacetimeDB is the heart of Oasis

SpacetimeDB gives us a database with a built-in state machine, so we are able to control the state of every single machine in the pool: the commander node and every helper node.

What we were able to do was create an orchestration system on top of it. We have one commander node and different helper nodes. The commander doesn't have to hold the full model in memory; the model can live on a bigger machine. The commander tells each helper what data it actually needs, sends it over the network, and it's done, like magic.

We've done this on a small scale, but the really cool thing is what it means for bigger institutions like hospitals and school systems. They're not bottlenecked by $10 Ethernet cables. There are miles of dark fiber already underground. Dallas was once a banking hub and Fort Worth was the transportation and railroad hub of the United States. We have the infrastructure, we have the fiber, and this is the technology to really connect all of our devices.

It's also extremely secure, because it runs on an isolated network, and we can query the database itself to see what's going on with every single machine. Right now we're running Gemma 4 26B across all three machines. The model needs about 25 GB for its weights, and the most usable GPU memory we have on one machine is about 17.8 GB. We're able to run this big model across three smaller machines.

This is just an elementary system. You could add another machine and make it your one dedicated agent: different Macs running different models, and one machine doing agent orchestration. Agents like Hermes or a Claude bot can run on another machine on the network, and the pool just computes for all of them.

And we built it in Rust. Rust is portable across Mac and Windows: install Rust and the needed dependencies, and you can connect your device to the network.

What lives in the database:

Tables: node, heartbeat, command, split_plan, cluster_status, event. Every machine is a row, every order is a row.
Reducers are the only way to change state, and the cluster state machine (EMPTY → PLANNED → STARTING_HELPERS → LAUNCHING → SERVING) is enforced in the database, so illegal transitions are rejected.
Agents and the coordinator subscribe to the tables, so every client sees each change live. There is no separate server or status API.

Oasis and legacy modernization

Whenever an organization wants to modernize a system, the data involved is often the most sensitive kind: healthcare, government, law, mortgages. Those records and that code can't be sent to a cloud AI. Oasis is the kind of infrastructure that lets smaller vendors take part in modernization: strong open models running on hardware they already own, on a private network, so sensitive data never leaves the building.

How we built it

The Oasis protocol is the control layer, written in Rust on a self-hosted SpacetimeDB module (2.14.1), compiled to WebAssembly, running on the main machine.
Each machine runs a Rust agent: it probes the hardware, registers with SpacetimeDB, heartbeats every second, and starts or stops llama.cpp's rpc-server on command.
A Rust coordinator reads the pool, plans the split, launches llama-server, and re-plans when a machine joins or drops.
Generated, typed SpacetimeDB bindings in a shared contracts crate, so a schema mismatch between the database and a program is a compile error.
The model math runs on llama.cpp (build 8665), which splits the model by layers across the GPUs and passes activations over its RPC backend.
Hardware layer: a GL.iNet Opal travel router, a 5-port gigabit switch and USB-C Ethernet adapters. The network stays offline; no prompt or document leaves the room.
Tools: netcheck (reachability and round-trip time), bench (time to first token, tokens/s) and a live dashboard in one HTML file.

Challenges we ran into

Venue Wi-Fi blocked laptop-to-laptop traffic, so the pool got its own router and switch.
The 26B stalled on the 16 GB main machine because llama.cpp mapped the whole 25 GB file into memory. Loading with --no-mmap makes main hold only its 5.7 GB share.
A sleeping laptop takes the pool down: llama.cpp aborts when a helper disappears. The coordinator now detects it, re-plans and recovers.
macOS Local Network privacy silently blocked the coordinator when it was launched from another app; it has to start from Terminal.
A 25 GiB model takes about an hour to download on venue Wi-Fi.

Accomplishments that we're proud of

A model too big for any of the laptops runs across three of them at 12.5 tokens/s, offline.
The whole pool is run through a database: start, join, model switch and recovery are rows and reducers.
A laptop joins a running pool and the model re-splits itself.

What we learned

On a laptop pool, latency sets the speed, not bandwidth: each token moves a few kilobytes per link. More machines add capacity, not per-user speed.
Mixture-of-experts models suit pooling: Gemma 4 26B reads about 4B of its 26B parameters per token.
SpacetimeDB works well as a control plane: agents call reducers and every client sees the same state live.

What's next for Oasis

Speculative decoding with a draft model on main, so one network round trip covers several tokens on every task.
Encrypted links between machines for regulated offices.
One-command setup: plug in the box, run one agent per machine, the pool forms.
Windows and Linux desktops in the same pool as Macs.