Inspiration
Post-harvest losses cost small-scale African farmers up to 30% of their harvest every season. The FAO has documented exactly how to prevent this — moisture thresholds, pest control, hermetic storage — but that knowledge sits in a 47-page English PDF, unreachable at the point of need for a farmer without connectivity who may ask in Swahili.
HarvestMind exists to close that gap. The question was simple: can a sub-4GB system, running on a $200 laptop with no internet, answer a farmer's storage question accurately enough to actually protect their harvest?
What it does
HarvestMind is a fully offline agricultural advisory system that answers post-harvest storage questions in English and Swahili, grounded entirely in the FAO Grain Storage Manual.
Ask it how to seal a storage container against insects — it retrieves the relevant manual sections and gives you the steps. Ask in Swahili about moisture damage in the granary — it detects the language, translates the query, retrieves the answer, and responds in Swahili. It never guesses: if the manual does not cover a question, it says so.
Everything runs locally. No API calls, no cloud, no connectivity required. Peak RAM stays under 4 GB on an 8 GB machine.
How we built it
The pipeline has five stages, each designed around the 7 GB RAM ceiling and CPU-only constraint:
Chunking — The FAO PDF is a two-column layout with infographic diagrams, tables, and yellow highlight boxes. We built a custom PyMuPDF parser using horizontal banding to reconstruct reading order across columns, producing 54 section-aware chunks with page and heading metadata.
Contextual enrichment — Before indexing, each chunk is prefixed with 1–2 LLM-generated sentences situating it within the document. This bridges the vocabulary gap between how a farmer asks ("plastic container") and how the manual is written ("Small containers", "hermetic bags"). Prefixes reach the index only — the generator never sees them.
Indexing — A hybrid index combining BM25 sparse retrieval (with a custom suffix stemmer — no NLTK dependency) and BAAI/bge-m3 dense embeddings in fp16. All vectors are L2-normalized at index time so dot products equal cosine similarity at query time. BM25 and dense are fused via Reciprocal Rank Fusion (k=60) with a section-diversity cap to avoid redundant chunks.
Retrieval — English queries use the full hybrid pipeline. Swahili queries use dense-only: BM25 receives noise from Swahili morphemes against an English corpus, so we measured the hybrid path actively hurting recall and disabled it for non-English input.
Generation — Qwen2.5-1.5B-Instruct (Q5_K_M GGUF, llama.cpp) with a manually constructed ChatML prompt. The model generates in English only — direct Swahili generation at this quantization degenerates into repetition regardless of decoding parameters. Swahili answers are produced by routing through NLLB-200-distilled-600M (translate query → generate in English → translate answer back).
Memory is managed by strict sequential loading: embedder loads, encodes, and is deleted before the LLM loads. The two large models are never resident simultaneously, keeping peak RSS in the 2–4 GB range end-to-end.
Challenges we ran into
The cross-encoder reranker consistently hurt accuracy. We planned a three-stage pipeline with a ms-marco-MiniLM cross-encoder for final reranking. After three independent ablation runs on our 18-question eval set, it degraded the correct top result every time — specifically breaking the plastic-container query that is one of the two mandatory test prompts. We dropped it. The literature recommends rerankers; our data did not. We kept the data.
Direct Swahili generation from a 3B model is unreliable at Q4 quantization. We tested every combination of temperature, repeat penalty, and prompt instruction. The model would begin a Swahili response then collapse into repetitive, semantically broken output. This is a capability limit at this quantization level, not a prompting issue. The translate-around-generation architecture solved it cleanly and also improved retrieval quality as a side effect.
A silent chunker bug deleted an entire section. When a PDF text block contained both a heading and its body text together, our parser was advancing the section pointer without saving the accumulated body text. The "Small containers" section — which contains the answer to one of the mandatory test prompts — was silently dropped from the knowledge base for several days before we caught it by auditing chunk IDs against the table of contents.
Thermal throttling on the reference hardware profile. The Qwen2.5-3B model at Q4_K_M measured 93–94°C and triggered the profiler's throttle flag on our test hardware, which maps to a −10 point penalty. Switching to Qwen2.5-1.5B Q5_K_M resolved the thermal issue while recovering quality through higher quantization precision — Q5 preserves more weight fidelity than Q4, partially compensating for the smaller parameter count when combined with retrieval grounding.
PDF two-column layout extraction. PyMuPDF returns text blocks in document order, not visual reading order. A two-column page would interleave left and right column text. We implemented a horizontal banding algorithm that groups blocks by vertical overlap, then sorts within each band by x-position, reconstructing left-then-right reading order reliably across all 47 pages.
Accomplishments that we're proud of
An 18-question held-out evaluation set spanning the full manual, tagged by difficulty and language, with an automated Recall@5 runner. Every architecture decision — model choice, retrieval strategy, reranker removal, embedding model — was made from measured evidence on this set, not intuition or the two mandatory prompts alone.
bge-m3 at fp16 achieves 94–100% Recall@5 on the eval set versus 78–83% for multilingual-e5-small, with zero measured accuracy loss from the fp16 quantization. This result held across three independent runs.
The pipeline stays under 4 GB peak RSS end-to-end on a machine with 8 GB total RAM, leaving enough headroom to run the OS, the profiler, and the full inference stack simultaneously without risk of OOM disqualification.
Found and reported two reproducible bugs in the competition's own reference profiler tool during testing.
What we learned
The gap between what the literature recommends and what your data shows is real and matters. Rerankers are consistently described as accuracy improvements in RAG research. On this specific domain, with this specific model, they consistently hurt. The 18-question eval set was what let us catch that — two data points (the mandatory prompts) would not have been enough.
Memory engineering on a hard RAM ceiling is architecture, not optimization.
Deciding which models are never resident simultaneously, in what order they
load, and verifying it with /proc/self/status checkpoints at each stage is
the difference between a system that fits and one that disqualifies itself
with an OOM crash.
Chunking quality is the foundation everything else rests on. A retrieval algorithm cannot return a chunk that does not exist. The silent deletion bug that removed "Small containers" from the index was invisible to every metric except manually checking whether the answer to the test prompt was actually reachable.
What's next for HarvestMind: Agri-Edge Intelligence
Contextual Retrieval at scale — The LLM-generated situating prefixes are already implemented and ready to evaluate. Anthropic's published results show 35–49% reduction in retrieval failure on structured documents. We expect meaningful gains on the heading-vocabulary gap this manual has.
RAFT-style LoRA fine-tuning — The recurring numeric-inversion failure (model restating a threshold with its direction reversed) is a distractor- susceptibility problem that prompt engineering only partially mitigates. RAFT fine-tuning via Unsloth, exported back to GGUF, targets this class of error directly using the existing GPU.
Additional FAO documents — The indexing architecture is document-agnostic. Adding a new PDF requires running one script. Expanding to cover post-harvest loss prevention, crop drying, and market-access guidance from the broader FAO library would dramatically improve hidden-prompt coverage.
Additional African languages — The translate-around-generation architecture extends to any language NLLB-200 supports, which includes Amharic, Hausa, Yoruba, Igbo, and 16 other African languages. The retrieval pipeline requires no changes.e Intelligence
