AfyaShield-LLM

Bilingual CPU-native LLM safeguarding East African healthcare by auditing claims, clinical notes, and metadata in English and Swahili to detect medical fraud offline.

Comment

Healthcare schemes across Africa suffer from staggering structural leakage. According to estimates by the Africa Centres for Disease Control and Prevention (Africa CDC), over 50% of available health resources continent-wide are lost to fraud, poor planning, and ghost systems.

The real-world financial toll is alarming:

  • Public Sector Interceptions (Kenya): Under Kenya's Social Health Authority (SHA), digital audits and tracking blocked over KES 12.7 billion in suspicious or fraudulent medical claims before payout, alongside ongoing investigations recovering hundreds of millions in confirmed losses (such as KES 278 million recovered).
  • Systemic Losses (South Africa): The Special Investigating Unit (SIU) notes that roughly 15% of total healthcare expenditure is lost annually to fraud, waste, and abuse, deflecting tens of billions of rands away from genuine patient care.
  • Private Underwriting Impact: Regional private underwriters, such as Old Mutual Holdings, report verified annual fraud losses exceeding KES 106 million, while industry bodies estimate that ~20% of all submitted private medical claims contain fraudulent elements or inflated charges.

We were inspired to build AfyaShield-LLM to put an offline-first, bilingual "digital gatekeeper" directly into the hands of local hospital auditors and field claims adjusters in East Africa running entirely on budget laptop CPUs without expensive cloud APIs.

What it does

AfyaShield-LLM is an edge-native, cross-disciplinary audit agent optimized for East African health insurance claims. Operating strictly within a 3.45 GB RAM footprint, it performs joint reasoning across:

  1. Bilingual Clinical Notes: Parses mixed English and Swahili clinical narratives (e.g., matching medical history against claimed procedures).
  2. Metadata Timeline Integrity: Detects creation/modification date manipulation and flags invalid authoring environments (e.g., claims created via graphics design suites instead of hospital EMR systems).
  3. Tariff & Demographic Validation: Audits requested payment amounts against local KES tariff schedules and verifies clinical compatibility against patient gender and age demographics.

How we built it

We fine-tuned Llama 3.2 3B Instruct on specialized health claim schemas, clinical audit logs, and English-Swahili medical translations. To enable local execution on resource-constrained hardware, we packaged and deployed the model using llama.cpp:

  • Quantization Strategy: Quantized to GGUF Q4_K_M (4-bit Medium), maintaining 6-bit quantization on critical attention layers while compressing weights to ~2.0 GB.
  • Pipeline Integration: Encapsulated within an idempotent Bash runtime (download_model.sh) paired with Python JSON schema validators.
  • Optimization: Fine-tuned context prefill buffers to prevent memory allocation spikes during multi-document input parsing.

$$S_{\text{score}} = 0.50 \cdot S_{\text{acc}} + 0.30 \cdot S_{\text{perf}} + 0.20 \cdot S_{\text{eff}}$$

Challenges we ran into

  • Context Latency on Pure CPU: Running prefill over 512 prompt tokens on a dual-core Intel Xeon @ 2.20GHz CPU yielded an initial first-token latency approx 65 seconds. We balanced batch evaluation sizes in llama.cpp to prevent thread thrashing and thermal throttling 0.0% throttling achieved.
  • Swahili Clinical Fine-Tuning: Translating complex medical fraud jargon into colloquial Swahili required careful alignment to ensure the model correctly caught implicit contradictions (e.g., mapping "Mgonjwa alikuja kwa ushauri wa afya ya ngozi" to an outpatient dermatology visit rather than surgical prostate care).
  • Memory Budget Constraints: Staying under the strict 7.0 GB competition RAM ceiling required tuning memory mapping (mmap) parameters to hold peak RSS at 3,449.77 MB (~3.45 GB).

Accomplishments that we're proud of

  • Budget & Multilingual Multipliers: Qualified for both the +10% Budget Laptop and +15% African Language multipliers (+25% total boost).
  • Strict Parameter Fit: Achieved 70.0% normalized accuracy on standard evaluation benchmarks (arc_easy) within a 3.2B parameter budget.
  • Zero Swap Overflow: Maintained steady-state operational RSS at 3,339.38 MB, using less than half of the permitted 7.0 GB threshold.

What we learned

  • Quantization Trade-offs: Smaller 135M parameter models save memory but suffer catastrophic reasoning degradation on cross-lingual tasks. A 3B model quantized to 4-bit medium delivers the optimal Pareto frontier for domain-specific edge AI.
  • Metadata Tells the Story: Fraudulent claims often reveal themselves in document authoring metadata before the text is even read.

What's next for AfyaShield-LLM

  • Quantized Speculative Decoding: Integrating a lightweight 135M draft model alongside the 3B target model to reduce first-token prefill latency on multi-core mobile processors.
  • Direct EMR Plugins: Packaging the runtime as a lightweight C++ shared library (libafyashield) for native integration into open-source openMRS platforms used across East African clinics.

Built With

Share this project:

Updates