Olutoju

• 

  Offline system overview: local llama.cpp and vision runtimes, ready corpus, five clinical agents, and zero network inference.

• 

  Longitudinal postpartum thread with evolving observations, evidence-linked responses, and 10.7% local context usage.

• 

  Source provenance: pathway-gated retrieval exposes WHO and Nigerian Federal Ministry of Health passages and document metadata.

• 

  Real model-initiated local tools: evidence.read and clinical_math complete before an approval-gated referral proposal.

• 

  State-changing referral.prepare action remains pending until an officer explicitly approves and executes the exact proposal.

• 

  Offline vision review: identity-free image, structured findings, unverified machine draft, and officer confirmation boundary.

• 

  Auditable reasoning workflow: local input safety, evidence retrieval, model-led review, structured validation, and approval gating.

• 

  Local audit trail records the clinical turn, referral proposal, model metrics, agent trace, vision draft, attachment, and thread creation.

Inspiration

Frontline health workers may need to assess postpartum concerns with unreliable connectivity, incomplete observations, and ordinary 8 GB laptops. A general chatbot can blur reported facts, source-backed guidance, and unsupported inference. Olutoju was built to keep those boundaries visible.

What it does

Olutoju is an offline maternal and postpartum encounter workspace for Nigerian nurses, midwives, and Community Health Extension Workers. It keeps identity-free encounter notes, observations, attachments, local evidence, referral plans, approvals, and audit history on the device.

The assistant separates recorded facts, retrieved clinical evidence, model inference, missing information, and uncertainty. It does not diagnose, prescribe, invent measurements or facilities, or treat an image as a confirmed clinical finding. High-risk and uncertain cases are escalated to the applicable protocol and qualified clinical review.

How we built it

Olutoju runs MedPsy-1.7B Q4_K_M through upstream llama.cpp with no cloud fallback during inference. A Next.js workspace provides the clinical workflow, while SQLite stores encounters, evidence provenance, approvals, and audit events locally. SQLite FTS5 retrieves twelve source-attributed passages from seven WHO and Nigerian Federal Ministry of Health sources.

Five versioned clinical-agent lanes divide intake, maternal review, guidance, safety, and handoff responsibilities. Eight local MCP tools expose evidence lookup, encounter reads, deterministic clinical arithmetic, and approval-gated state changes.

Challenges we ran into

The default llama.cpp host-memory prompt cache grew with every distinct conversation prefix and could eventually exceed the target laptop budget. We reproduced the growth, bounded the cache to 256 MB, and verified a flat eleven-turn soak. We also serialized the optional vision and clinical models so they do not remain resident together longer than needed.

Clinical safety required another kind of engineering: every model-generated plan is grammar-constrained, checked against deterministic policy, linked to local evidence IDs, and recorded in an audit trail.

Accomplishments

• Fully local inference with no network fallback
• Real GGUF packaging through an idempotent, checksum-verified download script
• Schema-valid full ADTC profiler run with no skipped accuracy stage
• 65.46 generation tokens/second and 2,371 MB peak RSS on the development machine
• Full-stack peak of 4,922 MB in the heaviest measured photo-review flow
• Eleven-turn stability soak at approximately 2,642 MB total stack memory
• Source provenance, uncertainty handling, and officer approval boundaries throughout the workflow

The published benchmark values above were measured on an Apple M3 Pro development machine, not the ADTC reference-class Ubuntu laptop, and are presented as development evidence rather than audit-hardware claims.

What we learned

On-device AI is a systems problem, not just a model-selection problem. Cache policy, process lifecycle, retrieval design, structured outputs, and honest uncertainty handling mattered as much as quantization. Narrowing the product to one real Nigerian frontline-health workflow made the offline constraint useful rather than decorative.

What's next

Next steps are a target-class Ubuntu audit run with complete thermal telemetry, independent clinical review, and iterative usability testing with qualified frontline health workers. Olutoju remains an early proof of concept, not a medical device or deployment recommendation.

Built With

• llama.cpp
• next.js
• vercel-ai-sdk