CoffeeVision: Offline AI for Coffee Disease Detection

An offline, on-device AI system that detects coffee leaf diseases and delivers treatment recommendations in English and Kiswahili, helping farmers reduce crop losses and improve yield.

Comment

Inspiration

Smallholder coffee farmers in Uganda and East Africa often lose yield when leaf diseases such as coffee leaf rust and Phoma spread before they can get expert help. Internet connectivity is unreliable and cloud AI is costly. We built CoffeeVision to run entirely on a commodity laptop so farmers and extension workers can diagnose leaves and receive advice without the internet.

What it does

CoffeeVision is an offline web application that lets a farmer:

  1. Upload or capture a coffee leaf photo
  2. Classify the leaf as healthy, Leaf rust, or Phoma (ONNX CNN, ~1 second on CPU)
  3. Get structured advisories with symptoms and countermeasures in English or Kiswahili
  4. Chat with CoffeeVision for follow-up questions using an on-device language model (GGUF + llama.cpp) Everything runs locally on an 8 GB RAM laptop with no cloud APIs during use.

How we built it

Trained a DenseNet121-based convolutional neural network on labeled coffee leaf images and deployed it as an on-device system optimized for offline inference on a HP EliteBook (Intel i5, 256gb,8gb ram, 2.20GHZ) Integrated an offline AI advisory assistant that allows farmers to ask questions and receive guidance on coffee diseases, prevention, and treatment, enhancing usability beyond image classification African Language Support: Supports English and Kiswahili, enabling farmers to understand diagnoses and treatment recommendations in their local context. This improves accessibility, trust, and adoption in real-world use.

Challenges we ran into

  • Two model types: CNNs classify images; LLMs generate text — they cannot be merged into one GGUF file, so we designed a hybrid pipeline.
  • 8 GB RAM: We chose SmolLM2-360M at Q4_K_M quantization to fit alongside the classifier on a budget laptop.
  • Kiswahili quality: The small LLM is English-biased; we added locale-based structured advisories and Kiswahili quality checks in chat.

Accomplishments that we're proud of

  • Achieved 83.88% accuracy in disease detection and successfully deployed the model to run without internet on standard machines.
  • End-to-end offline farmer workflow: photo → classification → advisory → chat
  • Bilingual UI and advisories for English and Kiswahili
  • Passes ADTC RAM constraints (~375 MB LLM peak RSS) on an 8 GB laptop
  • Built by a cross-disciplinary agriculture + AI student team at Soroti University, Uganda

What we learned

  • How to combine vision and language models on resource-constrained hardware
  • Designing for farmers: simple upload/capture flow, structured advisories, and conversational follow-up
  • ADTC submission requirements: GGUF + llama.cpp, offline inference, and profiler benchmarking

What's next for CoffeeVision: Offline AI for Coffee Disease Detection

  • Fine-tune SmolLM2 on agriculture and Kiswahili Q&A
  • Expand disease classes with more Ugandan field samples
  • Package for extension offices and explore lighter mobile deployment

Built With

Share this project:

Updates