Arise

🧟‍♂️ About the Project

💡 Inspiration

Modern AI is powerful, but it comes at a cost — high-end GPUs, expensive infrastructure, and massive energy consumption. At the same time, millions of perfectly usable laptops and desktops are discarded every year because they can’t keep up with modern software.

We asked a simple question:

What if we could bring these “dead” machines back to life and make them run AI together?

This idea led to Arise — a system that transforms old, unused hardware into a distributed AI supercomputer.

🛠️ What We Built

Arise is a distributed AI mesh that shards a language model across multiple devices and runs inference collaboratively.

Instead of one powerful machine, we use many smaller ones:

Each node processes a portion of the neural network and passes hidden state tensors to the next node, enabling true pipeline parallelism.

Key features:

🔗 Model sharding across multiple devices
⚡ Plug-and-play node addition
📊 Real-time monitoring dashboard
🧠 Tensor-level communication (not just text passing)
🔄 Dynamic layer rebalancing

🧠 How It Works

We split the model into layers:

Node 1 → Layers 0–4
Node 2 → Layers 4–8
Node 3 → Layers 8–12

Instead of sending text, nodes exchange hidden state vectors:

[ h_{i+1} = f_i(h_i) ]

Each device computes part of the model and forwards the result, creating a distributed inference pipeline.

⚙️ Tech Stack

Python
PyTorch + Transformers
FastAPI (worker nodes)
Streamlit (dashboard)
Socket + REST APIs (network communication)

🚧 Challenges We Faced

1. Model Compatibility

Most modern models (LLaMA, Gemma) are gated or require custom architectures, making integration difficult. We had to carefully select models that worked locally and adapt our pipeline.