• 

  The app that I've built to integrate the model.

• 

  Loading the model with llama.cpp

From Local Knowledge to Local AI: An Offline Agricultural Assistant for Africa

The Spark: A Farmer's Question in a Language No AI Understood

It started with a simple conversation.

During a visit to a farming community in southwestern Nigeria, I watched a smallholder farmer named Adeola struggle to diagnose a strange yellowing pattern on her cassava leaves. She had no internet access, no extension officer within reach, and no smartphone app that could understand her question in Yoruba. She showed me her field, described the symptoms in her native tongue, and asked: "Kí ni ohun tó ń pa ewé kásàvà yìí?" — "What is killing my cassava leaves?"

That question stayed with me.

I realized that the world's most advanced language models — trained on trillions of tokens — had never really been built for Adeola. They required stable internet, expensive API credits, and spoke languages she couldn't fully trust. Meanwhile, the knowledge she needed existed in academic papers, agricultural manuals, and extension guides — all locked behind digital walls she couldn't scale.

What if the AI could come to her, in her language, on the laptop she already had?

---

The Vision: AI for the Hardware Africa Actually Has

The Africa Deep Tech Challenge 2026 posed a deceptively simple question: Can you build an LLM that runs entirely offline on an 8GB laptop with no GPU?

Most AI research focuses on scaling up — bigger models, more parameters, larger clusters. This challenge asked us to scale down — to compress intelligence into constraints that millions of Africans already live with.

The ambition was radical but grounded: democratize access to agricultural knowledge by making AI work on the hardware Africa actually has.

---

The Build: From Dataset to Deployable Model

1. Choosing the Foundation

We started with DeepSeek-R1-Distill-Qwen-1.5B — a lightweight, multilingual model with strong performance on reasoning tasks and a license that allowed commercial deployment. Its 1B parameter count made it a natural fit for our 8GB RAM target, and its exposure to over 140 languages gave us a head start on multilingual capabilities.

2. The Dataset: 50,000 Agronomy Questions

We fine-tuned on Professor/agronomy-qa-pairs, a dataset of ~50,000 curated agricultural Q&A pairs covering:

• Crop management and planting techniques
• Pest and disease identification
• Soil science and fertilization
• Climate adaptation strategies
• Post-harvest handling

The dataset included examples in English, Yoruba, and Hausa — exactly the languages Adeola and millions like her speak every day.

3. The Fine-Tuning Challenge

Fine-tuning a 1B model on 50,000 examples would normally take hours of GPU time. Using Unsloth with QLoRA (4-bit quantization with LoRA adapters), we reduced memory consumption dramatically while maintaining accuracy.

The training ran on a T4 GPU, completing in approximately 6 hours — well within the constraints of a typical cloud notebook.

Key hyperparameters:

• LoRA rank: 64
• Batch size: 4 (with gradient accumulation)
• Learning rate: 2e-4
• Epochs: 2 (to avoid overfitting)

4. From Hugging Face to GGUF

The fine-tuned model needed to run through llama.cpp, the only runtime allowed by the competition. This required a two-step conversion:

1. Merge LoRA adapters into the base model
2. Convert to GGUF and quantize to Q4_K_M

The Q4_K_M quantization was a deliberate choice — it balanced:

• Size: ~750 MB (down from 2 GB)
• Speed: 8-12 tokens/second on CPU
• Quality: Minimal loss in accuracy compared to higher-precision formats

5. Building the RAG Pipeline

A pure language model, even fine-tuned, can hallucinate. To ground the model in verifiable facts, we built a Retrieval-Augmented Generation (RAG) pipeline:

• Vector database: ChromaDB with all-MiniLM-L6-v2 embeddings
• Knowledge base: 5 agricultural PDFs and 3 text files (473 chunks total)
• Retrieval: Top-3 most relevant documents injected into the prompt
• Fallback: Model uses its fine-tuned knowledge when no relevant documents exist

The RAG pipeline runs entirely offline — no API calls, no external dependencies.

6. The API: FastAPI + llama.cpp

To serve the model, we built a lightweight FastAPI application that:

• Communicates with the llama-server process (using the OpenAI-compatible API)
• Handles RAG retrieval and prompt construction
• Supports CORS for frontend integration
• Exposes a simple /perguntar endpoint

The frontend is a clean, mobile-responsive chat interface with:

• Session persistence (conversation history)
• Text-to-speech (using the Web Speech API)
• Voice input support (speech-to-text)
• Source attribution (shows which documents informed each response)

---

The Challenges: What We Learned

Challenge 1: The Tokenizer Trap

The most difficult problem was converting the fine-tuned model to GGUF. The Gemma tokenizer is based on SentencePiece, while the conversion script expected a GPT-2 style tokenizer. This caused an AssertionError during vocabulary mapping.

Solution: We used Unsloth's native save_pretrained_gguf() method, which handles Gemma's tokenizer correctly and produces a fully compatible GGUF file.

Challenge 2: African Language Hallucination

Initial tests in Yoruba and Swahili produced grammatically correct but factually wrong responses. The model was "hallucinating" because it lacked sufficient training data in these languages.

Solution: We added examples from the dataset in multiple African languages and included language-specific context in the RAG pipeline. The model now generates more accurate responses in target languages.

Challenge 3: Memory Management

The 8GB RAM limit was unforgiving. Even with a 750 MB model, the memory footprint grew during long conversations due to KV cache accumulation.

Solution: We limited context length to 1024 tokens and implemented auto-clearing of conversation history when memory usage exceeded safe thresholds.

Challenge 4: Making it Work Offline

Ensuring zero external network calls during inference was surprisingly difficult. The sentence-transformers model used for embeddings initially attempted to download files on first use.

Solution: We pre-downloaded all models and cached them locally, using the local_files_only=True flag for transformers and sentence-transformers.

---

The Result: A Model That Fits Africa

After weeks of iteration, we had a working system that:

• Runs 100% offline on an 8GB laptop with no GPU
• Speaks Yoruba, Swahili, Hausa, and Nigerian Pidgin
• Answers agricultural questions with RAG-backed accuracy
• Fits in ~750 MB (leaving plenty of room for other apps)
• Responds in 8-12 tokens/second (feels responsive to users)

Test Results

Language	Prompt	Response Quality
Yoruba	Kí ni àgbẹ̀?	Correct definition of agriculture
Swahili	Eleza kilimo cha usahihi	Accurate precision agriculture explanation
Nigerian Pidgin	Where we fit get bean seeds wey get iron content?	Practical, actionable advice
English	How does Brassicaceae suppress fungal disease?	Complete scientific explanation

---

The Impact: Who This Serves

This model is not just a technical artifact — it's a tool for real people in real situations:

• Adeola, the cassava farmer, can now ask about crop diseases in Yoruba and get expert advice instantly, without leaving her field.
• Samuel, a student in a rural school, can access agricultural knowledge for his assignments without paying for data.
• Fatima, a community extension worker, can take a $150 laptop into villages and train farmers using AI-generated, locally relevant content.

---

What I Learned

This project taught me that AI for Africa isn't about making bigger models — it's about making models that work with what we have. It's about compression, efficiency, and most importantly, inclusion.

Language should never be a barrier to knowledge. The internet should not be a prerequisite for expertise. And the most powerful AI in the world is useless if it doesn't speak the language of the people who need it most.

---

The Road Ahead

The submission deadline is August 25, 2026, but the work doesn't end there. I plan to:

1. Expand the knowledge base with more localized agricultural data for different regions
2. Add more African languages (Igbo, Amharic, Kinyarwanda)
3. Optimize memory usage further to allow longer conversations
4. Build a mobile app wrapper so farmers can run this on their phones
5. Integrate voice-first interaction for users with low literacy

---

Acknowledgements

• Africa Deep Tech Foundation for creating this challenge
• The unsloth team for making fine-tuning accessible
• The open-source community behind llama.cpp, ChromaDB, and sentence-transformers

---

The Code

All code is open-source and available at: https://github.com/r3tnuh-007/ADTC_2026

---

"The most powerful AI in the world is useless if it doesn't speak the language of the people who need it most."

Built With

• 1b
• 3
• cpu
• face
• fastapi
• gemma
• gguf
• google
• hugging
• inference
• llama.cpp
• lora
• peft
• python
• q4-k-m
• qlora
• quantization
• sentence
• transformers
• unsloth