VECTOR AFYA

• 

  ADTC profiler tests

• 

  Opening the application and typing a prompt

• 

  The response

• 

  Testing another prompt

• 

  How to proceed after an answer

Inspiration As a nursing student in Kenya, I have seen the gap firsthand and it's a real challenge, reliable healthcare information and nursing education support shouldn't depend on stable internet or an expensive cloud subscription. The ADTC "Laptop LLM Challenge" was a chance to prove that useful healthcare intelligence can run entirely offline and be accessible to everyone, on the same mid-range laptop most people already own with zero need for cloud, API fees, or connectivity requirement.

What it does VECTOR Afya is an offline first healthcare AI assistant. Users get to pick one of four modes — Clinical Information, Medical Q&A, Nursing Education, or Patient Education — or let it auto-detect. It has Assessment & Escalation,not a fifth category but a cross cutting safety layer that runs underneath all four, so a patient-education question about a stroke still gets flagged for urgency if it needs to be.The system routes the query, applies category-specific instructions, runs pre- and post-generation safety checks for red flags and unsupported diagnosis, and generates a response using a local Qwen2.5-3B-Instruct model through llama.cpp which all is fully offline, CPU-only, no internet required at inference time.

How I built it I forked the ADTC submission template, built out a dedicated vector-afya branch Evaluated Qwen2.5-1.5B, Qwen3.5-0.8B, Qwen2.5-3B, and Qwen2.5-7B against the hardware budget (Intel i7-6500U, 4 cores, 7.6GB RAM, no GPU) ,the 7B was ruled out on memory and so landed landed on Qwen2.5-3B-Instruct Q4_K_M GGUF which was the most promising. After coming up with the first architecture and general router around the model, I realised it would even perform better if I built a hybrid router with explicit user category selection is the default path (faster, no wasted classification call, no ambiguity), with auto-detect as a fallback,that was my best idea on this project. I built a layered prompt architecture — global healthcare instructions + category-specific instructions + safety rules — rather than one generic system prompt ,built safety.py as a genuine pre-generation (red-flag/urgency detection) and post-generation (checks for unsupported diagnosis, false certainty, missing escalation) layer .Afterwards,verified locally with the official ADTC profiler: ~2.8 tok/s generation, ~3.45GB peak RSS, no thermal throttling and added automated tests (pytest) from the start, not after the fact Used AI-assisted development heavily for implementation (I'm a nursing student, not a full-stack developer) — I owned the architecture decisions, category design, and safety rules myself, and made a point of personally reproducing every benchmark number before claiming it.

Challenges I ran into First-token latency of ~88 seconds is the biggest unsolved problem — I haven't fully isolated whether it's cold model load, prompt evaluation, or llama.cpp configuration,I'll look into it. CPU-only inference on 4 cores means real generation speed (~2.8 tok/s) is slow enough that a full answer can take over a minute — a genuine usability problem I haven't solved yet, not a hidden one. Being honest about what's actually built versus what's on the roadmap took discipline — it would have been easy to describe planned features like RAG or Swahili as if they already worked,nut my model allows for these to be implemented later on.

Accomplishments that I'm proud of A local LLM answering real nursing and health questions completely offline, on hardware people already own, with no GPU is a mile stone for me as a nursing student. A genuine layered architecture ,routing, category,specific prompts, and a cross-cutting safety layer , instead of a single prompt wrapped around a chatbot Passed the ADTC profiler with no thermal throttling and a reproducible model checksum Automated test coverage built in from day one

What we learned

3B model doesn't need to "know all of medicine" to be useful as I initially thought,the routing, safety, and eventually retrieval layers around it matter more. Perceived responsiveness streaming matters as much as raw throughput for something people are meant to trust Offline-first isn't just a technical constraint in this context ,Kenya's Digital Health Act and emerging AI-in-healthcare oversight mean this space has real regulatory weight beyond the hackathon, which changes how I think about what "shipping" this would actually require

What's next for VECTOR AFYA A persistent multi-question chat loop and response streaming, so it doesn't sit frozen for 60+ seconds A local browser UI so people don't need to run terminal commands to use it RAG over a curated, trusted medical knowledge base, so responses are grounded in real sources instead of parametric memory alone Swahili support built into the safety/escalation layer itself, not bolted on as a translation layer Real testing with actual nursing students before any patient-facing use,taking it step by step safety first.

Built With

• cpu-inference
• gguf
• git
• gith
• healthcare-ai
• linux
• llama.cpp
• offline-ai
• on
• pytest
• python
• qwen2.5