onara-ai-afrika-brain-offline

100% offline AI coding assistant powered by Qwen2.5-Coder via llama.cpp — runs on an 8GB laptop, no internet, no cloud required.

Comment 1

Inspiration

Millions of students and developers across Africa face unreliable, expensive, or completely absent internet access. Cloud-based AI coding assistants are effectively out of reach in these conditions. Onara.AI (Afrika Brain Offline) was built to solve exactly that: a coding assistant that runs 100% on-device, with zero dependency on the cloud, so useful AI help doesn't depend on having a stable connection.

What it does

Onara.AI is an offline AI coding tutor and developer assistant. It helps users write, debug, and understand code — explaining algorithms, catching syntax and logic errors, and providing structured refactoring — entirely offline, once the model is downloaded.

How we built it

  • Model: Qwen2.5-Coder-3B-Instruct, quantized to GGUF Q6_K for a strong balance between coding accuracy and resource footprint.
  • Runtime: llama.cpp / llama-cpp-python, running fully on CPU — no GPU required.
  • Backend: FastAPI, serving the local inference engine over a REST API.
  • Frontend: Flutter, giving a clean, cross-platform interface across desktop and mobile.

We chose Q6_K over a more aggressive quantization specifically because precision matters for a coding assistant — small errors in generated syntax are costly for this use case.

Challenges we ran into

  • Migrating the inference layer from an early Ollama-based prototype to a direct llama.cpp integration, to meet the competition's runtime requirement.
  • Development happened on an Intel i5 8th-gen laptop — below the competition's reference hardware profile — which led to observable thermal throttling under sustained inference. We documented this transparently in our REPORT.md, along with the mitigations we investigated.
  • Working across a Python/FastAPI backend and a Flutter frontend required careful structuring to keep the two in sync while iterating quickly.

Accomplishments that we're proud of

Getting a genuinely useful, fully offline coding assistant running end-to-end — model, backend, and GUI — on consumer-grade hardware within an 8GB RAM budget.

What we learned

How much of a difference quantization choice and thread configuration make on real consumer hardware, and how important it is to design for connectivity constraints from day one rather than as an afterthought.

What's next for Onara.AI

Exploring a lighter quantization fallback mode for lower-spec hardware, and packaging the app for easier distribution across more African markets. We're also planning to develop a natural language model for Angola's traditional languages — Kimbundu, Umbundu, and Kikongo — to make offline AI assistance genuinely accessible to speakers of these languages.

Built With

Share this project:

Updates