Vital-Med Tiny

About the project

Vital-Med Tiny is an offline health-information and wellness assistant for the Africa Deep Tech Challenge 2026 Laptop LLM track. It runs as a single GGUF file through llama.cpp on a normal CPU laptop, with no cloud API and no GPU requirement.

The project is built around a simple question: what happens when the internet, electricity, or API budget disappears, but people still need practical health guidance?

In many African settings, the device that survives the longest is not a cloud service. It is the laptop already on the desk, the machine that can keep running on battery while the router is down. Vital-Med Tiny is my attempt to put useful medical triage and wellness guidance inside that machine.

Inspiration

The ADTC challenge frames the bottleneck clearly: on-device AI should work on the hardware people already own. Healthcare made that constraint feel urgent. A student in Lagos, a clinic worker in Asaba, or a software developer in Nairobi should not need a powerful GPU or a paid API just to get basic, cautious health information.

I wanted the model to answer questions like:

• "My 3-year-old in Lagos has had fever for 2 days and is drinking poorly. What should I do now?"
• "I forget to drink water while coding all day. Give me a simple hydration and micro-break plan."
• "How should I measure blood pressure at home?"

The target was not to replace a clinician. The target was to build a small, local assistant that consistently says: here are likely possibilities, here is what to do now, here are red flags, and here is when to seek professional care.

This project is also inspired by Vitál: A Wellness OS for People Who Live at Their Laptops and serve as a addition to Vitál as a local LLM.

Vital As The Product Motivation

This project is also a continuation of my earlier Vital wellness assistant work: https://github.com/EddyEjembi/Vital and https://huggingface.co/blog/build-small-hackathon/vital-wellness.

Vital was built around the idea that health support should feel personal, local, and continuous: hydration, medication reminders, meal planning, exercise logs, symptom tracking, weekly summaries, and wellness coaching. But a cloud-dependent assistant has a weakness in the exact settings where local autonomy matters most: unstable internet, privacy-sensitive health data, and devices with limited compute.

Vital-Med Tiny is the local model layer I wanted Vital to have. Instead of only building a UI that calls a remote model, this submission asks whether the health assistant's reasoning core can live on the laptop itself. That is why the model was trained not only for medical Q&A, but also for wellness routines, ergonomic advice, hydration coaching, blood-pressure habits, and tool-call style outputs that match the kind of actions Vital already supports.

What I built

Vital-Med Tiny is based on Qwen3.5-2B and fine-tuned with LoRA SFT on a curated healthcare and wellness dataset. The final submitted model is:

• Runtime: llama.cpp
• Format: GGUF
• Quantization: Q4_K_M
• File: vital-med-Q4_K_M.gguf
• Selected run: qwen35-2b-sft-20260820-103431
• Primary language: English
• Domain: healthcare / medical triage / patient education / wellness coaching

The model is designed for offline use on the ADTC laptop profile:

$$ S_{total} = 0.50S_{acc} + 0.30S_{perf} + 0.20S_{eff} - P_{thermal} $$

That scoring formula changed the project. It was not enough to chase a bigger model. The right system had to be accurate enough, fast enough, and comfortably under the 7 GB RAM ceiling.

How I built it

The pipeline had five major stages:

1. Data assembly from real and synthetic healthcare sources.
2. Cleaning and preparation into supervised fine-tuning chat rows.
3. Modal GPU training with Unsloth and bf16 LoRA.
4. GGUF export and local llama.cpp testing.
5. Profiler and qualitative evaluation to select the final model.

The final SFT training mix included:

• real doctor-patient Q&A
• handcrafted and synthetic Vital-Med examples
• African-context health and wellness examples
• general instruction rehearsal
• medical MCQA rehearsal
• tool-calling examples
• clinical-note summarization examples

I tested three model paths:

Candidate	Decision
Original SFT Q4_K_M	Selected
Correction SFT	Rejected
Tiny DPO pass	Rejected

The original SFT was not perfect, but it was the most stable. The correction and DPO experiments fixed some local issues while damaging general response quality, which is a bad tradeoff for a healthcare assistant.

Challenges

The biggest challenge was not training a model. It was deciding when not to keep training.

Several things looked promising and then failed under evaluation:

• A correction SFT pass improved some surface behavior but made JSON and tool-negative behavior less reliable.
• A DPO pass fixed one declared JSON case but degraded everyday wellness answers.
• Larger 4B models were slightly better on arc_easy, but much slower and less competitive under the laptop constraints.
• African-language outputs were not reliable enough to make that the core claim, so the final package is deliberately conservative.

The hardest lesson was that "more tuning" is not automatically better. With a small model, a tiny dataset can oversteer the distribution quickly.

What I learned

I learned that edge-model work is systems engineering all the way down:

• model size matters
• quantization matters
• prompt format matters
• data cleaning matters
• chat template behavior matters
• evaluation quality matters more than a single benchmark number

I also learned to keep the submission honest. The profiler numbers are good, but healthcare quality is a human-facing concern. The final report documents both the successful path and the rejected experiments.

Final result

The selected model measured on my participant laptop:

Metric	Result
Generation speed	10.41 tokens/s
Peak RSS	2.01 GB
arc_easy proxy	0.68 acc_norm, 50 samples
Thermal throttling	No

This leaves large memory headroom under the 7 GB limit and keeps the model usable on CPU-only hardware.

What's next

The next version would focus less on adding more broad training and more on stronger evaluation data:

• clinician-reviewed red-flag cases
• better home-care thresholds
• safer pregnancy triage
• human-verified African-language examples
• more robust JSON-format adherence

Vital-Med Tiny is a small model, but the project proves the shape of the thing: useful health guidance can live on ordinary laptops, and the design discipline for that is different from cloud LLM work.

Built With

• lora
• modal
• qwen3.6
• sft
• unsloth