Mburu AI

• 

  palindrome demo

• 

  sum even demo

Inspiration

Not every CS student in Africa has stable internet. A university lab connection that drops mid-lecture, a hostel wifi shared by fifty people — for a lot of students, tools like GitHub Copilot or ChatGPT just aren't reliably usable, even though they'd genuinely help. That's not a skills gap, it's an infrastructure gap, and it felt like the right thing to actually build for instead of just work around.

What it does

Mburu AI is a desktop coding assistant that runs entirely offline. No cloud API, no subscription, no internet required after the model is downloaded once. It writes code, explains it, and debugs it — running Qwen2.5-Coder-3B-Instruct (GGUF Q4_K_M) locally through llama.cpp, on hardware as modest as 8GB RAM with integrated graphics only.

How we built it

The app itself is a PyQt desktop application with chat history and project-based context. The model runs through llama.cpp for CPU-only inference — the only runtime ADTC's evaluation pipeline accepts, and also just the right fit for integrated-graphics hardware with no discrete GPU. We quantized to Q4_K_M to keep peak memory around 3.2GB, well under the 7GB budget, while holding onto solid code quality. The system prompt and a low generation temperature (0.2) are tuned specifically to keep responses focused on correct, readable code rather than creative variation.

Challenges we ran into

Memory was the constant tension — every choice, from quantization level to context size, came back to fitting comfortably inside 8GB total RAM without sacrificing response quality. We also tested several inference-level optimizations (thread count, flash attention, quantized KV cache) and learned that this model/hardware combination had already hit its natural throughput ceiling — a good reminder that not every bottleneck can be tuned away, and that being honest about that in the report matters more than chasing marginal numbers.

Accomplishments that we're proud of

Getting a real, usable coding assistant to run entirely offline on modest hardware — not a stripped-down demo, but something that actually helps with real debugging tasks, like catching a logic bug in a function and explaining why it was wrong.

What we learned

That model selection and quantization strategy matter as much as raw model size — a well-chosen 3B model, properly quantized and prompted, can genuinely hold its own on real coding tasks without needing a bigger, heavier model or any GPU dependency.

What's next for Mburu AI

Retrieval-augmented generation over local course material and documentation, so students can get help grounded in their actual curriculum. Also exploring quantization tradeoffs further to see if we can push throughput higher without giving up code accuracy.

Built With

• gguf
• llama.cpp
• pyqt
• python
• qwen2.5-coder