Inspiration
Smallholder farmers across Africa often can't get fast, reliable answers when a crop starts wilting or a flock of chickens starts dying — the nearest agricultural extension officer or vet might be hours away, and a bad guess (or a lucky one from an ill-informed shopkeeper) can wipe out a season's income. At the same time, the "AI advisor" answer to this problem usually assumes a cloud connection and a subscription — neither of which is a safe bet on a rural laptop with patchy or no internet. We wanted to see how far a genuinely offline, free, runs-on-a-basic-laptop advisor could go if we took the 8GB-RAM, no-GPU constraint seriously instead of treating it as a demo footnote.
What it does
FarmOS is an offline farm advisory assistant. A farmer (or an extension worker on their behalf) types or speaks a description of what they're seeing, optionally attaches a photo, and gets back grounded, actionable advice — not a generic guess, but a diagnosis backed by a retrieval-augmented reference corpus covering real crop and poultry conditions.
- Text or voice input — describe the problem in plain language, or record it and let offline speech-to-text (whisper.cpp) transcribe it.
- Photo diagnosis — attach a photo and two specialized, fine-tuned classifiers run alongside a general fallback model: a 38-class crop disease classifier (14 species: apple, tomato, grape, potato, maize, and more) and a poultry disease classifier trained on fecal images (Coccidiosis, Newcastle disease, Salmonella, Healthy).
- Grounded advice, not hallucination — every answer is retrieved from a hand-written reference corpus (RAG via ChromaDB) before the LLM (Qwen2.5-1.5B-Instruct, quantized GGUF, running through llama.cpp) generates a response, and the sources are shown alongside the advice.
- Read-aloud — answers can be read back using the browser's built-in speech synthesis, for low-literacy users or hands-busy situations.
- Farm record history — every diagnosis is logged locally (SQLite) with an optional plot tag, so a farmer can track recurring issues on a specific field or coop over time, and export the history as CSV or plain text.
All of it — LLM, retrieval, both vision classifiers, speech-to-text — runs entirely offline, sequentially, on a standard 8GB-RAM laptop with no GPU.
How we built it
- LLM: Qwen2.5-1.5B-Instruct, GGUF Q4_K_M quantization, served through llama.cpp — the only LLM runtime path, per the challenge's rules.
- RAG: ChromaDB (onnxruntime-backed embeddings, no PyTorch/network dependency at runtime) + llama-index for document loading and chunking, over a hand-written markdown corpus of crop and poultry conditions.
- Vision: MobileNetV2 transfer learning — the ImageNet-pretrained feature extractor frozen, only the classifier head retrained — for both the crop classifier (5,700 PlantVillage images) and the poultry classifier (600 images selectively extracted from a public Zenodo fecal-image dataset via HTTP range requests, avoiding an 8.5GB full download). PyTorch is a dev-only fine-tuning tool; the shipped app runs every model through onnxruntime only.
- Voice: a prebuilt whisper.cpp binary (
tiny.en), invoked as a subprocess so its memory is fully released the moment it exits. - Memory discipline: nothing runs concurrently. The LLM, each vision classifier, and whisper.cpp are loaded fresh, used, and explicitly freed before the next component runs — verified with real RSS sampling, not assumed.
- Frontend: plain HTML/CSS/JS, no build step, served directly by FastAPI.
Challenges we ran into
The most instructive one: after expanding from a single 4-class maize classifier to two separate specialized classifiers (38-class crops, 4-class poultry), we discovered a shared confidence threshold that worked fine for one classifier actively broke in two different directions for two. The 38-way crop model spread its confidence thin even when it was right — a correctly-identified tomato blight photo scored only 30% — while the 4-way poultry model concentrated confidence easily, including on the wrong domain, confidently misnaming a plain crop-leaf photo as "coccidiosis" at 51-73%. We caught this by testing real held-out photos end-to-end rather than trusting the code, split the threshold per classifier, and added a rule that when both specialized models fire confidently on the same photo, only the more confident one is trusted. We documented what's still unsolved (a genuinely unrelated photo can still fool the poultry model) rather than quietly hiding it.
Beyond that: keeping total RAM under budget as we added components one at a time, working around a Windows console-encoding crash during model export, and sourcing a usable poultry dataset at all — there's no public "sick chicken photo" dataset with reliable ground truth, so we deliberately scoped that classifier to droppings, a real diagnostic signal already used in poultry husbandry, and said so clearly in the UI.
Accomplishments that we're proud of
- A crop classifier covering 38 real disease/healthy classes across 14 species at 92.46% validation accuracy, and a poultry classifier at 89.17% — both trained from real public data, not synthetic or assumed.
- Peak memory across the entire pipeline (LLM + all three vision classifiers + retrieval), measured with real RSS sampling, sits at ~1.98GB — comfortably inside an 8GB budget with room to spare.
- Every claim in our technical report is something we actually tested, including the ones that turned out to be wrong on the first attempt.
What we learned
Confidence scores from a softmax classifier are not free honesty — a model with no "none of the above" class will always hand you a confident-looking number, and the only way to know if that number means anything is to feed it real photos and watch it fail. Tuning for one classifier's failure mode doesn't transfer to a differently-shaped classifier for free.
What's next for FarmOS
Multilingual support (the architecture is already language-agnostic — corpus, embeddings, and the LLM itself all support it, only the current content is English-only), livestock beyond poultry, and a real fix for the wrong-domain overconfidence problem — most likely a lightweight learned "background" class or an upstream domain classifier, rather than more threshold-tuning.
Log in or sign up for Devpost to join the conversation.