Sahel Agri — an offline agronomic advisor that refuses to guess
Agricultural extension in the francophone Sahel runs on paper. The technical content farmers need — sowing calendars, seed densities, fertiliser doses, striga and fall-armyworm control, variety catalogues — exists as PDFs published by RECA Niger, INRAN, ICRISAT and CGIAR centres. It reaches the field through an extension agent carrying a phone, in an area with intermittent connectivity and intermittent electricity.
The question asked in the field is narrow and quantitative: "quelle dose d'engrais par poquet ?", "quand semer le mil à Maradi ?" A wrong number there is not a neutral error. It is a season of fertiliser bought and misapplied.
What made me build it this way
Before writing any retrieval code, I measured what the naked 0.5B model does on exactly these questions: 18 generations, 3 candidate quantizations × 6 French agronomic prompts, greedy decoding, single seed.
- 0 correct agronomic facts out of 6.
- 0 refusals out of 18 — although every prompt carried the explicit instruction "si tu ne sais pas, dis-le".
- Worst measured case: the model prescribed 100 g of NPK per planting hole against 6 g in the source document. A factor of 16.7.
That measurement decided the architecture. A 0.5B model asked to be careful is not careful, so the refusal cannot be a line in a prompt — it has to be code that runs before the model is called.
How it works
Three answer levels, all decided in code upstream of the model:
| Level | Trigger | Behaviour |
|---|---|---|
| Sourced | the fact is in the corpus | verbatim quote + publisher + year + page number |
| Signalled | agronomic, but outside the corpus | useful answer, explicitly labelled unsourced, and it situates the gap — it names the closest documented value and how far it is from what was asked |
| Refusal | retrieval distance below threshold | says it does not know |
Retrieval. 33 agronomic documents → 677 pages → 3 180 chunks, each carrying full provenance (publisher, year, page, licence regime). Dense retrieval with BGE-M3 embeddings, run through llama.cpp like everything else, fused with BM25 (19 877 terms) by reciprocal rank fusion. The index is pre-built and committed — 14.6 MB — so the first launch is genuinely offline, with no first-run download.
BM25 is not decoration. On the millet-sowing question, dense retrieval alone ranks a cowpea manual first, because it does not know that a rare toponym like "Maradi" is the discriminating token. That is exactly BM25's job.
Cross-lingual by design. One of the two test prompts asks in English and is answered from a French source document, with the citation kept in French and attributed. The corpus is 19 French and 14 English documents; a fact proved absent in one language is not absent from the corpus, which is a lesson I learned the hard way (see below).
Measured results
Final profiler run — participant mode, official non-AVX image, 2 threads, --memory=7.5g, third of
three consecutive runs:
| Measured | Score | |
|---|---|---|
| arc_easy acc_norm, n=300, seed 42 | 59.33 % | S_acc 59.33 (declared proxy) |
| Generation throughput | 26.52 t/s | S_perf 100.00 (capped) |
| Peak RSS | 761.49 MiB = 0.74 GiB | S_eff 89.38 |
| First-token latency | 11 391 ms | — |
| S_total 77.54 |
The profiler builds this field as rss / (1024**2), so it is MiB and converts to GiB by 1024, not by
1000 — a detail worth one line of code reading, because getting it wrong moves S_eff.
Throughput is capped: 11.52 t/s of measured speed convert to exactly zero points. I ran the profiler three consecutive times rather than once, because generation throughput carries 8–18 % intrinsic dispersion at 2 threads and the audit tolerance is 25 % — a single run is a draw, not a measurement. Range across the three: 25.34–26.52 t/s on throughput, 761.49–763.45 MiB on memory, accuracy identical at all three.
The submission image is published by digest and pulls anonymously:
docker pull ghcr.io/mechakc/adtc-agri@sha256:e5f5d7ce461c2f9435ee11171f2693582e57fc71c1c90ecf7c2213c102e8e4cd
The hard parts
No AVX. The audit image is built with baseline SSE, which costs roughly 5.9× on prompt processing. Any figure measured on a natively-optimised build is not comparable, so every number here was measured inside the official image.
Licences live in the PDF, not on the website. One publisher's site says "help yourself"; page 3 of its own PDF says "all rights reserved, even partial". I read the licence page of all 39 candidate documents and dropped the ones I could not clear. Seven documents carry a confirmed CC BY-4.0 statement inside the file; the rest are indexed for citation with publisher, year and page, and the regime is declared in the report rather than glossed over.
One document was excluded by measurement. A broken font renders 34 % of its characters as fused
tokens — afatoxin appears 325 times against aflatoxin 13. Repairing it would have meant guessing
between real alternatives (the same font writes four where flour is equally plausible), which
would have manufactured false verbatim quotes. It is excluded, and the exclusion is recorded with its
measured reason.
Two-column PDFs read across the page. The default extraction glued a word from the left column onto a word from the right. I only adopted the fix after proving the multiset of non-space characters was byte-identical before and after — the flag changes order, never content.
What I learned
The most transferable lesson had nothing to do with models. A verification command whose failure looks like a plausible result is worse than no verification at all. I hit this five separate times: a shell pattern that matched every line and reported 89 carriage returns in a file that had zero; a tool that confidently declared a directory ignored by citing a blank line; a loop detector that rejected a healthy run because it was matching the build's own help text. Each one returned a believable number.
The rule I now follow: to check that something is protected, run the action the protection is supposed to block and confirm it is refused. Do not ask a tool to describe its intention.
The second lesson: I proved a fact absent from the corpus by searching in French only — inside the very project whose differentiator is cross-lingual retrieval. The fact was there, written in English. An absence claim has to name the languages it swept, not just the files.
What not to trust
The report ships 11 known limitations, each with the figure that bounds it. Three worth naming here:
- The accuracy proxy is a declared bet. arc_easy is English multiple-choice reasoning, not agronomic correctness, and no public agricultural validation set exists for this domain. The one corroboration I have is an ordering, not a correlation.
- The refusal gate measures retrieval distance, not domain. An off-topic question sharing vocabulary with an agronomic corpus can clear it: a request for a poem about the moon scores 0.4898 against a 0.4376 threshold. It then receives the signalled level — nothing is presented as verified — but that is not a refusal, and I do not claim the gate recognises a domain.
- The declared environment is a development workstation, not the target laptop class. The figures are this machine's, measured inside the official image.
Full detail, with every figure re-derived from the committed run rather than copied: REPORT.md
