EdgeIMCI

Inspiration

The motivation for this project was using technology to improve living conditions, particularly those of children in Africa. Child mortality is one of the strongest indicators of a country's health and development, and many preventable childhood deaths occur in settings where access to experienced clinicians, reliable connectivity, and advanced computing infrastructure is limited.

The WHO Integrated Management of Childhood Illness (IMCI) framework already provides frontline health workers with structured guidance for assessing and managing sick children. Our question was whether modern AI could make that guidance easier to use at the point of care without turning the system into an unreliable medical chatbot or making it dependent on the cloud.

Edge IMCI was therefore conceived as an offline-capable clinical decision-support system for frontline primary healthcare workers: one that can understand naturally written clinical findings while keeping the actual clinical rules explicit, deterministic, and auditable.

What it does

Edge IMCI allows a frontline health worker to describe a child's assessment naturally rather than manually completing a rigid digital form. A small language model converts those findings into a structured clinical encounter. The deterministic Edge IMCI engine then handles assessment completeness, IMCI classifications, urgency, referral, treatment and management logic. The model also supports a conversational interface for questions and partial assessments, but this is deliberately separated from the authoritative clinical decision path. This gives us two complementary modes:

• structured extraction, where the model converts clinical language into machine-readable evidence for the deterministic engine; (the clinical support software is build to constrain the model to this mode)
• free-form interaction, where the model can respond conversationally while remaining bounded by Edge IMCI's supported scope. (this is accessible from chatting with the downloadable weights)

The system is being designed to run locally on modest hardware without requiring internet access.

How we built it

Working with our clinical domain expert, Dr. Ayotunde Adegboyega, a former WHO Africa public-health professional with more than 20 years of experience in maternal and child health and African health systems, we translated a bounded portion of the WHO IMCI Chart Booklet into deterministic, machine-readable clinical logic for children aged 2 to under 60 months. Around this clinical core we built:

• a small-model structured extraction layer;
• explicit extraction modes for separating clinical extraction from ordinary conversation;
• deterministic completeness, classification, urgency, referral and management logic;
• synthetic clinical-data generation grounded in the deterministic engine;
• routing and scope-boundary behavior for unsupported or ambiguous requests;
• systematic semantic-state generation rather than relying only on paraphrased examples;
• evaluation infrastructure for comparing different model sizes, fine-tuning runs and checkpoints. For one major dataset expansion, we decomposed the supported clinical logic into 280 distinct semantic states, including danger signs, individual clinical branches, branch-complete assessments, holistic encounters, incomplete assessments and contradictory evidence. Each state can be represented both as a structured extraction task and as a free-form clinical interaction, allowing us to test whether the model preserves the same underlying clinical meaning across different interfaces.

Challenges we ran into

The hardest problem was not generating fluent text. It was preserving exact clinical meaning. Small differences such as:

• “able to drink” versus “unable to drink”;
• “blood in stool” versus “no blood in stool”;
• a current convulsion versus a previous history of convulsion;
• an explicitly absent finding versus a finding that was never assessed; can completely change the clinical interpretation.

We also found that incomplete encounters are not all the same. Sometimes there is enough information to answer one clinical branch even though the overall assessment is incomplete. In other cases a missing observation prevents a classification. And sometimes an already-known danger sign requires immediate action even before the rest of the assessment is finished.

Another challenge was preventing a small model from either answering questions outside its intended scope or becoming so defensive that it refuses legitimate IMCI cases. This led us to build explicit routing, scope-boundary and free-form behavioral evaluations alongside the structured extraction tests.

Accomplishments that we're proud of

We built an offline-oriented architecture in which the language model does not have to be trusted with the final clinical rules. The system can interpret naturally expressed findings while clinical decisions remain deterministic and traceable to the underlying IMCI logic. We also developed a much stronger evaluation process. This includes:

• a reviewed 78-case holistic clinical golden suite covering complete, incomplete and clinically significant assessments;
• systematic structured-extraction evaluation;
• semantic-state and minimal-pair testing for clinically important boundaries;
• dedicated evaluation of free-form clinical responses;
• tests for scope errors, inappropriate refusal, unsupported advice, negation errors and hallucinated findings;
• checkpoint-level model comparison so that improvements in one capability can be checked for regressions in another. Rather than relying only on validation loss, we now evaluate what the model actually generates.

What we learned

The most important lesson was that a language model does not need to be the clinical decision-maker in order to make a clinical system substantially easier to use. Language models are useful for dealing with the messy interface between people and structured software: natural language, incomplete descriptions, different phrasing and conversational interaction. Deterministic software is much better suited to enforcing exact clinical rules. We also learned that model quality cannot be measured with a single accuracy number. Extraction accuracy, negation handling, partial assessments, scope boundaries, conversational behavior and safety can fail independently. That has pushed us toward evaluation-driven development: train a model, expose its specific failure modes, improve the relevant data or system component, and test again.

What's next for Edge IMCI

The immediate next step is to use our evaluation system to compare several small model sizes and fine-tuning checkpoints and determine how much model capacity is actually required for reliable offline performance. We are also expanding the training and evaluation data around partial assessments, clinical boundaries, negation and free-form interaction, while preserving the deterministic engine as the source of clinical truth. Beyond the current clinical scope, we plan to extend the deterministic IMCI implementation to additional assessment areas for children aged 2 to under 60 months and eventually to the other IMCI age group, young infants aged 0–2 months. We also see voice as potentially the simplest interface for using Edge IMCI in practice. Rather than asking a health worker to navigate a large digital form, the worker could simply dictate the child's assessment as they work. Local speech-to-text would produce a transcript that can be reviewed or corrected before being converted into structured clinical evidence and passed through the same deterministic IMCI pipeline. Our aim is for this interaction to remain offline and usable on inexpensive hardware. The longer-term goal is therefore not simply a larger language model, but a more complete, rigorously evaluated and voice-accessible clinical tool that fits naturally into frontline workflows while remaining auditable and capable of running at the edge. I particularly like “Rather than asking a health worker to navigate a large digital form…” because it explains why voice matters, rather than just listing speech-to-text as a feature.

Built With

• finetuning
• modal
• python
• qwen