ZombieLLM

🧟 Inspiration

The GPT-OSS family of models are powerful and groundbreaking, but they’re too big to run on small edge devices. In a real survival scenario-whether that’s a zombie apocalypse or simply being offline-you can’t rely on giant servers. We wondered if an older model could be brought back to life, carrying some of that modern knowledge in a smaller body. That’s the idea behind ZombieLLM: a reanimated GPT-2 with just enough brains to be useful when resources are scarce.

🧟‍♂️ What it does

ZombieLLM reanimates GPT-2 with the brain of GPT-OSS-20B.
We distilled and compressed the giant model’s wisdom into GPT-2 XL, making it:

• Lightweight enough for laptops and Raspberry Pis
  
• Smart enough to hold its own in Q&A and instruction tasks
  
• Fully offline-because when the servers fall, your AI shouldn’t
  

☣️ How we built it

Every zombie needs a necromancer - ours was GPT-OSS-20B. We reanimated GPT-2 in three steps:

1. Data distillation - we used Dolly-15k and Alpaca as question sets, but instead of their original answers, we asked GPT-OSS-20B to generate distilled responses. This became a clean, high-quality dataset used to fine-tune GPT-2 XL with DoRA.
   
2. Knowledge distillation - custom TRL pass where the student aligns with the teacher on response-span representations (pooled hidden states) using cosine loss in a shared projection, alongside the usual cross-entropy loss.
   
3. Personality injection - a final fine-tune on survival + zombie persona data to give it character.
   

Finally, we compressed the model into GGUF and published it on Ollama, so it runs locally - even on a Raspberry Pi ZombieDeck.

⚔️ Challenges we ran into

• Handling tokenizer mismatches between GPT-OSS and GPT-2 (left-padding was our fix)
  
• Preventing over-compression that left the model sounding like a “lobotomized zombie”
  
• Training within limited GPU hours while keeping results strong
  
• Adding personality without drowning out factual accuracy
  

🏆 Accomplishments that we're proud of

• Successfully reanimating GPT-2 into a working, portable instruct model
  
• Showing that distilled knowledge from GPT-OSS-20B can live inside a 1.5B model
  
• Running ZombieLLM offline on edge hardware like a Raspberry Pi
  
• Turning a technical project into a memorable story that blends AI and survival humor
  

🧠 What we learned

• Distillation is tricky-small details like padding can make or break quality
  
• Edge-first AI is real: with the right tricks, even old models can be reborn
  

🔮 What's next for ZombieLLM

• Growing the undead family by reanimating other models with modern knowledge
  
• Building plug-and-play survival kits for offline AI on low-power hardware
  
• Experimenting with RLHF (Reinforcement Learning from Human/Zombie Feedback)
  
• Opening a community repo so anyone can join the reanimation lab
  

🧟 Reanimation is all you need. And the end is just the beginning…

Built With

• accelerate
• llama.cpp
• ollama
• openai-harmony
• peft
• pytorch
• safetensors
• sfttrainer
• tensorboard
• tokenizers
• transformers
• triton
• trl