IMCI Triage Decision Support

Inspiration

Nigeria adopted the WHO's IMCI (Integrated Management of Childhood Illness) protocol in 1997, but adoption on the ground tells a harder story: a Kano State study found fewer than 25% of first-level health facilities had community health workers actually trained in it. A separate digital clinical-decision-support pilot in the same state measurably improved protocol adherence once introduced. That's the gap this project targets — not a hypothetical use case picked to fit a rubric, but a documented, sourced problem: health workers in low-connectivity clinics who need to correctly apply a known, published protocol, on hardware that has no guarantee of ever seeing the internet.

The ADTC brief's own framing — "the 8GB laptop with integrated graphics, the machine already sitting on millions of desks" — is exactly the deployment reality for a rural Nigerian clinic. That alignment is what pulled this project toward Healthcare & Medical over the other six domains.

What it does

An offline, CPU-only decision-support tool that helps a community health worker correctly apply the WHO IMCI protocol for children 2 months to 5 years old. Given reported symptoms, it walks the same danger-sign-first, structured-classification logic a trained health worker would follow — general danger signs, then cough/difficult breathing, diarrhoea/dehydration, fever, and ear problems — and produces a classification (severe / moderate / mild) with the protocol's recommended action.

This is decision support, not diagnosis. It helps a health worker correctly apply a known, published algorithm. It is not a source of independent medical judgment, and does not replace the official chart booklet, training, or professional care. That boundary is stated explicitly in the code, not just in this description.

How we built it

Model: Qwen3.5-0.8B, Q4_K_M GGUF quantization (~533MB), CPU-only inference via llama.cpp. Chosen over a heavier hybrid architecture after real benchmarking showed large TPS/RAM margin at this size — no need to pay the complexity and risk cost of a bigger model or a two-model routing setup.

A verified ground-truth engine, not vibes: rather than trusting an LLM to recall clinical thresholds correctly from pretraining, the classification logic itself is a deterministic rule engine, built directly from the official WHO IMCI Chart Booklet (March 2014, ISBN 978-92-4-150682-3) — fetched, read, and cited line-by-line against the source rather than approximated from memory. For a life-relevant classification task, correctness should come from code matching a published standard, not from a small model's learned weights alone.

Hierarchical reasoning, honestly scoped: the ADTC brief calls for hierarchical/dual-timescale reasoning. Rather than force that architecture to also make the medical classification (an unacceptable safety risk for a 27M-parameter model), it's scoped to what it can safely and usefully do: adaptive question-asking — deciding what to ask a caretaker next, given answers so far, stopping the moment enough is known. A "slow" strategic layer and a "fast" step-execution layer, trained via imitation learning on trajectories generated by an expert policy that's itself derived directly from the rule engine's own branch logic — one source of truth, not two independently-maintained versions of "correct."

Challenges we ran into

Two real bugs, both caught by testing before they reached anything that mattered. First: an early version of the dehydration-severity logic checked two specific signs as if both were required, when the actual WHO protocol requires any two of four. That's a meaningfully different (and more clinically sensitive) rule — the bug would have missed genuine severe-dehydration cases presenting with a different pair of qualifying signs. Caught by a scoring script that unexpectedly failed 2 of 5 test vignettes, not by inspection.

Second, and more interesting: after building a full imitation-learning pipeline for the adaptive question-asking policy — synthetic data generation, a trained baseline model, and a two-level validation harness — the trained model hit 99.8% accuracy on individual next-question predictions but only 86.7% end-to-end classification accuracy. That gap was the real finding: a policy could look almost perfectly correct action-by- action while still occasionally producing the wrong final answer, because early mistakes cascade. Chasing that gap down led to a genuine bug in the expert policy itself — chief-complaint-based question reordering could cause the system to stop on the wrong symptom category before checking a higher-precedence one, even though the underlying rule engine always evaluates categories in a fixed order regardless of what's asked first.

Worse: an existing unit test had asserted that buggy behavior as correct, without ever checking it against what the rule engine actually returns on full information. It was only caught because the end-to-end validation number was low enough to be suspicious. After the fix — and after expanding the test suite's randomized fuzzing to actually vary the dimension that had caused the bug — the same pipeline hit 100% end-to-end classification accuracy on 1,000 fresh, unseen synthetic cases.

A late strategic correction: partway through, it became clear that the ADTC evaluation harness runs the submitted .gguf file directly through llama.cpp — it doesn't invoke a custom orchestration or tool-calling layer. That reframes everything above from "the shipped system" to "verified training-data infrastructure": the real next step is using the rule engine as a source of fine-tuning data so the base model's own weights reflect protocol-correct answers, since the deterministic safety net isn't something the grader ever actually calls.

What we learned

That validating at the right level of abstraction matters more than validating thoroughly at the wrong one. High accuracy on an intermediate metric (next-question prediction) hid a real problem that only end-to-end outcome checking surfaced. And that a test suite can encode a wrong assumption as "expected" just as easily as application code can — tests aren't automatically ground truth just because they're green.