Imagine you want to try out the newly released Kimi K3.1 Pro, a 3T parameter model, but you have PTSD from a leaked API key last week and your dad said no to spending $500K on 8x Nvidia B200 GPUs. Darn it. What do you do now?

Hypercompute brings multiplayer to AI inference by pooling compute, memory, and storage across friends’ computers. It allows for running open-source models too big to be run any single, ordinary laptop, by pooling and assigning model layers across GPUs and CPUs, managing weight downloads and caching, and coordinating inference over encrypted peer-to-peer connections. You can now ask your friends to bring the hardware, and Hypercompute makes it work together.

Inspiration

At DubHacks last week, I was trying to run a local model. Around 24 billion parameters, my laptop started overheating and throttling. Looking around, I noticed plenty of other people with laptops. What if I could just pull in everyone's laptops and somehow, magically, have the compute power to locally run trillion parameter models? Is this even possible?

That became the idea behind Hypercompute. Let people contribute part of their computers to a shared inference system.

What it does

Hypercompute takes a model and builds a deployment across the available machines and compute power/resources:

Join a pool: Friends can host or accept a private invite, and choose how much memory to contribute.
Plan the placement: The host checks model sizes, memory budgets, and optimises layer distribution. across several machines.
Load each machine’s share: Managed peers can download their assigned tensors directly and cache them for later runs.
Run distributed inference: Each machine executes its assigned layers. The host coordinates RPC calls and transfers intermediate activations between pipeline stages.
Use one interface: Chat through the dashboard or connect through the Python SDK.

The dashboard shows participating machines, deployment status, and the planner’s layer allocation. If a required peer disconnects, the affected run automatically checkpoints, stops, replans, and restarts the run.

How I built it

I designed Hypercompute around two connected problems. 1) Where model layers should execute, and 2) where their weights should live. Different machines have different strengths. A gaming laptop might offer fast CUDA compute but limited VRAM, while a MacBook contributes unified memory through Metal. Our architecture coordinates those resources through distributed layer execution, tiered memory management, and placement decisions that account for the network between them.

The backend uses Python and FastAPI to manage the pool. Each joining machine is read for its hardware and contribution limits, then passes an RPC handshake before becoming eligible. The host handles invitations, memory reservations, worker processes, and deployment state. I separated this control plane from the inference traffic so that joining, planning, loading, and generating each have a clear lifecycle.

For placement, I built a cost-based optimizer that searches contiguous layer boundaries, peer combinations, and execution order using dynamic programming. It considers tensor sizes, KV-cache requirements, compute buffers, storage throughput, and estimated communication costs. CPU and GPU allocations sharing the same physical memory are counted against one budget. The planner can also use measured profiles to replace estimates. More available memory gives us more options, but the network still gets a vote.

Our main distributed path assigns transformer layers across CPU, CUDA, and Metal devices. The host coordinates execution through RPC, with intermediate activations moving between pipeline stages. When the host has a GPU, I keep the output head there so the much larger vocabulary-sized logits stay local. Iroh carries control messages and RPC traffic through encrypted QUIC connections, with NAT traversal and relay fallback.

I also made weight distribution selective. The host reads GGUF metadata to identify the offsets and sizes of the tensors assigned to each peer. Managed peers fetch those byte ranges directly, verify them, and store them in a reusable cache. This reduces the amount the host needs to upload and avoids making every participant download the entire checkpoint.

For models that exceed accelerator memory, Hypercompute supports several offloading strategies. Some layers can execute from host RAM, while supported mixture-of-experts layouts keep selected routed experts in file-backed host memory. I added platform-specific native extensions too with bounded CPU weight streaming through reusable buffers on Windows, and memory-mapped CPU tensor caching on macOS. These give us explicit control over weight residency instead of relying entirely on the operating system’s paging decisions.

Finally, I connected the system to a React + TypeScript dashboard, double-click join scripts, and a Python SDK. Users can inspect the layer plan, deploy a model, watch its loading state, and stream responses. The host checks worker participation and monitors peer health throughout the run. If a peer machine disappears, it stops the affected deployment so it can be replanned and restarted.

Why it matters

Speed and efficiency: On average, my 'cluster' of a gaming laptop + two MacBook Airs delivered 2.28x faster generation on average and about 97% fewer SSD reads per token.
Environment and Sustainability: Combining available compute and memory to run larger models, reducing additional hardware purchases, cloud rentals, and electricity costs.
Privacy and Data: Ability to run inference on computers you choose, with encrypted communication between trusted peer machines.

Accomplishments that I'm proud of

I was able to run a trillion parameter model (Kimi K2.8 1T) across a GX10, a gaming laptop, and two MacBooks. This would have originally required at least 16 × NVIDIA H800 / H200 GPUs.

I also accomplished a gaming laptop and two macbook airs running a 120B parameter model: gpt-oss-120b at ~9 tokens per second.

I also turned the setup into join scripts, a dashboard, and an SDK. Getting distributed execution working mattered. Making it usable without a pile of terminal commands. This is what I am most proud of.

What I learned

The biggest lesson was more computers does not automatically mean more speed. Pooling helped most when it replaced expensive SSD reads with work on another machine. When the host already ran a model efficiently, network overhead could outweigh the benefits.

I also learned to measure prefill, decode, and cold-start time separately. A model can produce its first token sooner without generating subsequent tokens faster.

What’s next for Hypercompute?

This project reinforced my love and passion for distributed systems and inference engineering. I want to pursue Hypercompute as a personal research project because there are still many unanswered questions about distributed inference.

The experiments and benchmarking carried out in this hackathon made the next set of questions very explicit and clear. I was able to successfully get trillion parameter models locally running. However, exceptions like DeepSeek 671B decoding more slowly in some configurations as peers were added, and Kimi K2.8 (1T) showing improvements in prompt processing and time to first token with addition of peers was so bewildering yet awe-inspiring. Understanding how model architecture, layer placement, memory bandwidth, and network latency interact is something I want to explore further.

More technically, I want to further simplify installation, reduce cold-start time, and improve recovery through automatic replanning when a peer disconnects. The goal is to make running a model across friends’ computers feel as simple as joining a session, with placement decisions grounded in optimization and measurements.