MedSLM

Give every clinician a scribe: spoken consultations turned into structured shareable medical records, offline, bilingual, and faster than handwriting.

Comment

Inspiration

Being the uncle to 4 kids, I'm often the one dealing with hospital visits when they fall sick, and every time it means buying a new hospital booklet, even at a hospital we've consulted at before. I end up being the one explaining their past medical history to a new doctor, when a proper record should have let that doctor see it themselves and diagnose faster and more confidently.

My father passed away 8 years ago from meningitis. His illness went undiagnosed for too long as he moved between hospitals, each one starting close to blank, because his past records weren't shared or easily obtainable across facilities. By the time the right exam was finally suggested and we learned what was actually killing him, it was too late. That loss is the real reason this project exists: to make it possible to produce and retrieve structured medical data quickly, so information travels with the patient between specialists instead of dying with each visit.

As a university student who works with AI models, I also see the deeper technical gap behind this: most medical data in Cameroon still exists only as handwriting in paper booklets, unusable for anything beyond the single consultation it was written for. Turning consultations into structured digital documents — without adding to a doctor's workload — doesn't just solve my family's problem. It's also a first step toward being able to analyze population health at scale, from real facts instead of paper nobody can search.

What it does

MedSLM is an offline documentation assistant for clinicians working with no reliable internet connection. A clinician dictates or types a consultation in ordinary spoken French — informal, incomplete sentences, the way they'd actually talk while examining a patient — and the tool produces a properly structured clinical document: a compte-rendu de consultation or a lettre de sortie, following real hospital document conventions rather than an invented format. The structured document is generated by a 3B-parameter language model (Ministral-3-3B-Instruct) running entirely on-device via llama.cpp, grounded through local RAG over the target hospital's own report-format rules, so the same tool can be reconfigured per facility without retraining anything.

Alongside the French professional document, the tool produces a patient-facing Swahili summary for cross-lingual handoff, built from a Swahili-French/English medical dictionary and fixed sentence templates — not free model generation — so the Swahili side carries minimal hallucination risk even though we cannot yet fully verify open-ended Swahili output ourselves.

The output is a printable, structured document meant to go straight into the existing paper record — faster and more complete than a rushed hand-written note, and, longer-term, exportable into a shared record a colleague or specialist could actually search and reuse.

How we built it

The core pipeline is: dictation → llama-server (OpenAI-compatible /v1/chat/completions endpoint) → RAG-grounded system prompt (format rules only, never patient facts) → structured French document → deterministic dictionary substitution → Swahili summary.

We evaluated several small models before settling on Ministral-3-3B at GGUF Q2_K quantization: SmolLM2-1.7B drifted off-task in French; Qwen3.5-0.8B burned its whole token budget on hidden chain-of-thought reasoning; Luth-1.7B (a French Qwen3 fine-tune) reliably refused the document-generation task itself, and on the runs where it did comply, it leaked raw RAG context and format instructions directly into the clinical output — a second, distinct failure mode, and arguably worse than an outright refusal because a document that looks complete but contains contaminated content is a real risk in a clinical setting. Llama-3.2-1B hallucinated a worked example from the RAG corpus into the actual patient's letter. Ministral-3B at Q2_K was the smallest model tested that reliably separated "format guidance from RAG" from "facts from the user" and did not fabricate or refuse.

We initially served the model with per-turn llama-cli calls and regex-extracted the reply from CLI output; this was unreliable because whether a raw -p flag is applied as a real system-role message or just prepended completion text depends on the exact build. We switched to llama-server's HTTP API with real role-separated JSON messages and no output-scraping, and kept the server warm across turns to avoid reload cost.

For the African-language side, we chose Swahili as our reference language largely because it's one of the most widely documented African languages, giving us a realistic base of dictionary and reference material to work from. We deliberately rejected free-generating Swahili with Ministral, since neither of us can independently verify Swahili medical text and Ministral's non-European-language coverage is thin. Given our hardware constraints, further fine-tuning the 3B model on Swahili wasn't feasible either. Instead we built Swahili sentence templates plus a Swahili-French/English dictionary, filled from the same structured facts already extracted for the French document — a narrower but low-hallucination-risk approach. During the evaluation phase, we plan to get human validation from Swahili speakers to confirm the correctness and accuracy of the dictionary and templates before treating them as fully verified.

Challenges we ran into

  • Model selection was mostly a process of elimination through failure, not success: three of the four alternatives we tried each failed in a different, model-specific way (refusal, chain-of-thought token exhaustion, context-leakage, hallucinated facts), and each failure mode had to be diagnosed separately before we could conclude it was a model problem rather than a prompting or pipeline problem.
  • Distinguishing thermal throttling from memory-pressure stalling. An early test of a higher-quality quantization (Q4_K_M) appeared to make the system grind to a halt on a RAM-constrained machine; without a temperature sensor reading at the time, we can't be certain whether that was CPU thermal throttling or heavy swap thrashing (we later found a comparison machine sitting near 94% swap use during a similar test) — these look identical from the outside but need different fixes, and we don't want to overclaim a specific cause we didn't measure directly.
  • Prompt-communication bugs disguised as model limitations. The llama-cli extraction pipeline silently corrupted what we treated as "the model's answer" for a while, which briefly looked like a model instruction-following failure until we isolated it as a serving-layer problem via side-by-side comparison against Ollama.
  • Keeping the Swahili output honest. It was tempting to just let a second small model generate Swahili freely, or run RAG over a Swahili dictionary and let the model compose from retrieved chunks — but retrieval only fixes what gets fetched, not what a model does with it once retrieved. We had to hold the line at pure lookup-and-substitute, no generation call at all, on a language we can't yet fully fact-check ourselves.
  • No real hospital template yet. Our current French document structure is an illustrative placeholder, explicitly flagged as such in our own corpus-generation script, not yet validated against an actual facility's real format — this remains an open item.

Accomplishments that we're proud of

  • A working, fully offline, single-3B-model pipeline that reliably distinguishes RAG-provided format rules from user-provided clinical facts, after ruling out four other models that each failed at this in a different way.
  • Minimal-hallucination-risk bilingual output: the Swahili patient summary is produced using a dictionary and template approach rather than free generation — a deliberate trade of coverage breadth for correctness in a language we can't yet fully verify ourselves.
  • Thermally safe operation on hardware weaker than the competition's reference spec: 75°C peak core temperature, not throttled, well under the 85°C cap, on a 2013-era dual-core CPU with no discrete GPU — reconfirmed across repeated runs.
  • Diagnosing and fixing a serving-layer bug (CLI-based prompt communication) that was silently masquerading as a model capability problem, by isolating it through a side-by-side comparison against a known-working serving path.

What we learned

  • A model "failing" at a clinical documentation task can mean several genuinely different things — refusing outright, generating but leaking irrelevant context, hallucinating from example data, or burning its budget on invisible reasoning — and each needs a different fix. Lumping them together as "the model doesn't work" would have led us to the wrong solution.
  • The safest way to add a language you can't personally verify isn't a smaller or different model — it's giving the model a series of Swahili-based templates plus a Swahili dictionary, so responses stay grounded in checked data rather than free generation. The model's job becomes structured-fact extraction; the dictionary does the actual language work.
  • Symptoms that look like one hardware constraint (thermal throttling) can actually be a different one (memory pressure) with an overlapping presentation, and self-reported dev-machine numbers don't reliably transfer across machines with very different RAM headroom — something we only caught by comparing two different test machines side by side.
  • Building for real clinical use surfaces requirements a generic benchmark doesn't capture: our own arc_easy sanity-check score says nothing about whether the model can correctly draft a compte-rendu a specialist would trust, which is why we kept our two required test prompts specific to real dictated clinical scenarios rather than generic Q&A.

What's next for MedSLM

  • Replace the placeholder French document templates with sourced, validated formats from an actual partner facility, ideally more than one, so the tool generalizes past our own illustrative examples.
  • Re-benchmark against hardware specs matching actual target hospitals, not just our dev machines, to confirm real-world effectiveness and efficiency.
  • Expand the Swahili dictionary's coverage beyond the current fixed phrase set.
  • Get human validation from Swahili speakers on the dictionary and templates, and validate document accuracy against real clinicians' judgment, not just our own read of the output, before broader deployment.
  • Build a web app giving the medical specialist control over execution modes, outputs and output formats, listening vs. speaking phases, and hospital-specific context.
  • Explore a structured local storage layer so a clinician's generated documents can be searched and reused for the kind of longitudinal, cross-hospital record continuity that motivated this project in the first place — turning single documents into a queryable patient history over time.
  • Test on cheaper, more portable hardware (Raspberry Pi-class single-board computers) in case a target hospital's existing computers are too outdated to run the current setup.

Built With

Share this project:

Updates

Submission history