MUFASA
Models for Understanding the Frontiers of African Scientific Advancement
A small foundation model that knows what African science has already found — and can reason with you about what it hasn't.
Ask most language models what has been studied about groundwater in the Cross River Basin, or which local materials have been tested as partial cement replacements in Nigeria, and you get plausible prose assembled from a global average.
MUFASA answers from a corpus of 10,480 African research papers it was actually trained on, names the study, and tells you when it doesn't know.
It runs offline, on a laptop, in about 800 MB.
Model: DestinyOtto/mufasa-gemma3-1b-sft-gguf
GitHub: github.com/Otto-Destiny/mufasa
At a glance
| Corpus | 10,480 African research papers, 6 domains, 2000–2026 |
| Extraction | 10,131 papers processed into 8 structured tables |
| Structured facts | 479,143 QA pairs · 661,786 evidence spans · 265,833 observations |
| Training set | 378,005 supervised examples (353,697 train) |
| Base model | unsloth/gemma-3-1b-pt — 1.0 B parameters |
| Stage 1 | Continued pretraining — rsLoRA r=128 |
| Stage 2 | Full-parameter SFT — all 999,885,952 parameters trainable |
| Deployment | GGUF Q4_K_M, ~800 MB, CPU-only, 7 GB RAM laptop |
| Dependencies at inference | none — no retriever, no index, no network |
Inspiration
Many capable AI systems assume access to powerful GPUs, large amounts of RAM, reliable broadband, and continuous cloud infrastructure.
Those assumptions do not always match the environments where scientific assistance could be valuable across Africa.
We started MUFASA with a simple question:
Can we build a useful scientific-reasoning model that runs locally on affordable CPU hardware?
We did not want to optimize only for benchmark accuracy. For a locally deployed model, usefulness depends on several factors at once:
Practical Utility = Accuracy + Throughput + Low Latency + Low Memory Use + Offline Accessibility
A model that is slightly more accurate but several times slower, larger, or dependent on cloud infrastructure may be less useful in practice than a carefully adapted compact model.
MUFASA therefore began as both a model-selection problem and a deployment problem.
The problem underneath it
African research output is real, growing, and largely invisible to the models people actually use. It sits in journals that are poorly indexed, in PDFs that are badly scanned, behind licences that vary paper by paper.
A researcher in Nsukka asking about local groundwater chemistry gets an answer shaped by the global literature — which is to say, mostly not about their problem.
Three consequences:
- Findings are re-derived because nobody knew the study existed.
- Local alternatives are overlooked in favour of imported defaults.
- The models get worse over time as global corpora grow faster than African ones.
MUFASA is a small, deliberate correction: take the literature that exists, structure it properly, and put it into a model small enough that anyone can run it.
What it does
MUFASA is a compact scientific-reasoning model that operates locally.
It is both closed-book and open-book
Most domain models are one or the other. A retrieval-augmented assistant has no knowledge of its own — take away the database and it is an empty shell. A fine-tuned model has knowledge but cannot cite, ground, or be corrected by a document you hand it.
MUFASA is trained on both behaviours, from the same corpus, in the same weights:
| behaviour | the prompt contains | the model must | provenance emitted |
|---|---|---|---|
| Closed book | study descriptor + question | recall from its weights | LEARNED_STUDY |
| Open book | evidence quotes + question | read and ground in the evidence | PROVIDED_EVIDENCE |
| Abstention | a question with no support | say so plainly | — |
The same extracted fact generates both modes. That is deliberate: a model that has only seen grounded answering never learns to recall, and one that has only recalled never learns to defer to a document.
Every answer carries the same contract
<answer>
Provenance: LEARNED_STUDY | PROVIDED_EVIDENCE | UNVERIFIED_STUDY | GENERAL_INFERENCE
Citation: (Author et al., Year)
Study basis: discipline; design; population; location; period
That is not decoration. It is what makes a small model safe to use in research: you always know whether it is recalling, reading, or inferring, and you always have enough to go and check.
It knows the African scientific landscape specifically
The corpus is not "science, filtered for Africa". Every paper passed a written relevance protocol in which author affiliation explicitly does not count:
affiliation_only— the only African signal is authorship, institution, venue, or affiliation country → excludeNever infer geography from names, affiliations, journal, DOI, language, or model familiarity.
A Nigerian professor's paper on generic corrosion chemistry is excluded. A study of cassava peel ash as a cement extender in Ogun State is included.
What the corpus captures is research about African materials, populations, environments, and constraints — which is precisely the knowledge a global model averages away.
What that enables
- Literature-gap reasoning — it has read the adjacent work and can say what has and hasn't been tested
- Local-alternative reasoning — 1,801 of the sampled training pairs are tagged
INNOVATION, covering local material substitutions, locally-sourced species and data, and constraint-driven engineering choices - Landscape awareness — which institutions, regions and disciplines have produced what
- Offline operation — no internet, no API, no data leaving the machine
The final model is distributed in GGUF Q4_K_M format and runs through llama.cpp. Once the model has been downloaded, no cloud inference API is required.
How we built it
OpenAlex — African science, 2000-2026
|
v
Frontier-LLM relevance classification
| |
included excluded (894 / 1,656 in benchmark)
| |
v dropped
PDF download and parsing
|
v
10,480 markdown papers (423 MB)
| |
v v
Three-task extraction Corpus split
CONTEXT / OBSERVATIONS train 9,806
/ TRAINING eval 337
| test 337
v |
8 Parquet tables |
479,143 QA pairs v
| Stage 1: continued pretraining
v rsLoRA r=128
Training-set funnel |
evidence / grounding / v
citations merged 16-bit CPT model
| |
v |
378,005 supervised examples |
open-book + closed-book |
+ refusal |
| |
+----------> Stage 2: full-parameter SFT
|
v
GGUF Q4_K_M — about 800 MB
Stage 0 — Model selection
Before training MUFASA, we benchmarked multiple compact and mid-sized language models using the Africa Deep Tech Challenge profiler components.
Candidate families included Qwen, Phi, Gemma and LiquidAI LFM, compared on ARC-Easy accuracy, generation throughput, first-token latency, peak process-tree memory, CPU efficiency, and an estimated weighted deployment score.
The Gemma 3 1B IT candidate achieved approximately:
| Metric | Result |
|---|---|
| Generation throughput | 17.4 tok/s |
| First-token latency | 2.54 s |
| Peak process-tree RSS | 1.33 GB |
| ARC-Easy accuracy | 0.50 |
| Estimated weighted score | 0.713 |
The 0.713 weighted score was the highest in that comparison. These are base-model selection measurements, not the final benchmark of the fine-tuned MUFASA checkpoint.
The benchmark gave us confidence that the Gemma 3 1B size class offered enough reasoning capability while leaving substantial memory headroom for local retrieval and application logic.
We then started the training pipeline from unsloth/gemma-3-1b-pt — the pretrained checkpoint rather than the instruction-tuned one, so that our own SFT stage would do the instruction tuning rather than layering a domain on top of someone else's.
Part I — Data engineering
Sourcing, and the relevance question
Papers were discovered through OpenAlex, filtered to African — predominantly Nigerian — science across 2000–2026. The hard question was not finding papers but deciding what counts as African research.
The naive filter is institution country. We rejected it, and wrote a protocol that says so explicitly. The classification runs on title and abstract only — authors, affiliations, DOI, journal and country codes are deliberately withheld from the prompt, so the model cannot cheat by recognising a Nigerian university.
Each paper receives a structured verdict:
| field | values |
|---|---|
evidence_level |
direct · inherent · latent · affiliation_only · absent · contradicted |
african_centrality |
0–4 |
hard_exclusion_reason |
affiliation_only · explicit_non_african_scope · outside_scientific_scope · … |
On a 1,656-paper labelled benchmark:
| outcome | papers | share |
|---|---|---|
| exclude | 894 | 54.0% |
| include | 670 | 40.5% |
| review | 92 | 5.6% |
54% of candidate papers were excluded. The single most common reason was affiliation_only — African authorship with no African research content. That number is the cost of taking the definition seriously, and it is why the corpus is what it claims to be.
A useful property emerged: african_centrality is strongly bimodal — 725 papers at 0, 644 at 4, only 102 across the whole middle. The judgement is usually easy; the protocol's nuance is spent on a small minority.
Corpus composition
10,480 papers survived classification, download and parsing.
| domain | papers | share |
|---|---|---|
| Health (HLT) | 3,838 | 36.6% |
| Environment (ENV) | 2,550 | 24.3% |
| Agriculture (AGR) | 2,401 | 22.9% |
| Energy (ENR) | 787 | 7.5% |
| Materials (MAT) | 599 | 5.7% |
| Technology (TEC) | 305 | 2.9% |
| total | 10,480 |
Every paper carries provenance in its front matter — OpenAlex ID, DOI, licence, PDF hash, parser used, and a family_id marking near-duplicate studies from the same group.
Extraction
Each paper went through a three-task extraction producing eight linked tables:
+--> CONTEXT --> study_contexts, paper_profiles,
| african_innovation
paper markdown --+--> OBSERVATIONS --> observations, evidence_spans
|
+--> TRAINING --> training_pairs, evidence_spans
|
+--> entity_mentions
| table | rows | what it holds |
|---|---|---|
entity_mentions |
663,518 | named entities linked to evidence |
evidence_spans |
661,786 | verbatim quotes with character offsets |
training_pairs |
479,143 | question / answer / reasoning |
observations |
265,833 | measured statements with comparison groups |
study_contexts |
40,897 | design, population, period, location |
extraction_status |
10,131 | per-paper audit |
paper_profiles |
10,016 | discipline, contribution, science checks |
african_innovation |
9,935 | local materials, substitutions, constraints |
Every claim is anchored. An answer is only usable if it points at a character span in the source document — which is what makes the grounding checks in the next section possible at all.
Engineering decisions worth naming
Resilience over strictness. Extraction ran through five gateways across thousands of papers. A single malformed JSON response should never cost a paper, so the pipeline salvages truncated output, mends stray quotes, retries while keeping the fullest attempt, and isolates failures per record.
JSON is the authority, Parquet is a cache. The tables are derived artifacts and were rebuilt from raw JSON when a writer bug was found. A staleness check compares JSON file count against table rows.
Schema drift is measured, not assumed. 393 papers marked complete produced no training pairs. Inspection found 123 with content under drifted keys (examples, qa_pairs, reasoning_examples) — 3,879 recoverable pairs, 0.81% of the corpus. The other 270 held no payload at all. Quantified, then deliberately deferred.
Author-year citations are resolved against the document, not just metadata. OpenAlex supplies a candidate; the paper's own first page is then scanned for publication-year and first-author signals. Where they disagree, the document wins:
| resolution status | papers |
|---|---|
| VERIFIED | 4,837 |
| METADATA_ONLY | 2,436 |
| CONFLICT | 2,006 |
| CORRECTED | 1,201 |
57% were confirmed or corrected against the paper itself. CONFLICT means the document offered two contradictory years at equal confidence, not that it disagreed with OpenAlex.
Part II — Building the training set
The funnel
Raw extraction is not training data. Everything passes a staged funnel, and every stage reports what it removed and why.
479,143 raw pairs
|
v
evidence recovery — six span shapes read, not one
|
v
script + language gates
|
v
study-family deduplication
|
v
grounding check — numbers, units, entities
| | |
supported context too narrow unsupported
| | |
| widen evidence QUARANTINE
| |
v <----------+
OPEN BOOK ---> closed-book rendering
| (study descriptor, no evidence)
| |
+------------+
|
v
citation attachment
|
v
378,005 examples
Reading evidence in six shapes instead of one raised coverage from 73% to 92% — recovering 88,242 pairs that were present but unread.
What the examples cover
Tag distribution across a 20,000-example sample:
| category | examples |
|---|---|
| FACTUAL | 8,528 |
| REASONING | 7,611 |
| QUANTITATIVE | 7,606 |
| INNOVATION | 1,801 |
| METHOD | 1,738 |
| CONCEPT | 1,537 |
| LIMITATION | 1,071 |
| ARGUMENT | 834 |
| MECHANISM | 815 |
INNOVATION at 1,801 is the category that carries local-alternative reasoning — substituted materials, locally-sourced species, constraint-driven engineering.
Splitting — done once, read everywhere
10,480 papers
|
v
group by family_id
|
v
stratify by domain
|
v
floor of 25 per domain per held-out split
|
+--> train 9,806
+--> evaluate 337
+--> test 337
|
v
audit: near-identical titles across splits
|
v
2 twins healed back into train
Two design choices worth stating:
A flat percentage keeps shares right but not counts. Technology is 2.9% of the corpus, so 3% of it is nine papers — enough for a pooled number, far too few to say anything about technology. A floor of 25 per domain per split costs 52 training papers and makes every domain individually scoreable.
The audit found two near-duplicate titles across splits — the same groups reporting adjacent work — and moved both to train rather than dropping them.
Part III — Model engineering
Why two stages
gemma-3-1b-pt (raw pretrained)
|
| Stage 1 — continued pretraining
v
knows the literature
|
| merge
v
16-bit CPT model
|
| Stage 2 — full-parameter SFT
v
knows how to answer about it
|
| quantize
v
GGUF Q4_K_M
The two stages have different responsibilities:
Continued Pretraining → Learn the scientific domain
Supervised Fine-Tuning → Learn how to answer users
CPT puts the facts and the vocabulary into the weights. SFT teaches the contract — question answering, provenance, citation, abstention.
Stage 1 — Continued pretraining on African research
| setting | value | why |
|---|---|---|
| base | unsloth/gemma-3-1b-pt |
pretrained, not instruction-tuned |
| method | rsLoRA, r=128, α=32 | rank-stabilised scaling (α/√r) |
| targets | q,k,v,o + gate,up,down + embed_tokens, lm_head |
182 LoRA modules across 26 layers, plus 2 full wrappers |
| window | 4,096 tokens | measured p99 of the corpus |
| corpus | train split only | evaluate/test never seen |
Making the token embedding layer and language-model head trainable mattered. Adapting to African scientific literature involves more than changing reasoning layers — the model also needs to adapt to vocabulary, names, geographic references, technical terms, and domain-specific language patterns.
Text preparation mattered as much as hyperparameters. Papers average around 10,000 tokens, and TRL truncates rather than windows. Measured, that would have discarded 28% of the corpus and the tails of 59% of papers — which is where discussion and conclusions live. Explicit tokenizer-aware windowing was added, with EOS on the final window of a paper only.
That last detail is the one we would defend hardest. A naive pipeline splits every paper into blocks and appends EOS after every block, which teaches the model that an arbitrary 4,096-token boundary means a document has ended. We instead maintained ordered windows and introduced EOS only at true document boundaries.
References were stripped: roughly 22% of a paper, almost entirely author names and years, and training on it teaches citation-shaped text with no way to be right.
Validation loss over the run:
| step | validation loss |
|---|---|
| 100 | 1.8207 |
| 200 | 1.7940 |
| 300 | 1.7802 |
| 400 | 1.7695 |
| 500 | 1.7622 |
| 600 | 1.7561 |
| 700 | 1.7523 |
| 800 | 1.7496 |
Monotonic, no overfitting, converging by step 855.
Stage 2 — Full-parameter supervised fine-tuning
Full fine-tuning, not LoRA. The reasoning: at 1 B parameters the memory argument for LoRA disappears (~10 GB of optimizer state), the CPT stage was already low-rank so a second low-rank pass compounds that limitation, and this stage is doing instruction tuning from a raw pretrained base — a large behavioural change, which is what full updates are for.
| setting | value |
|---|---|
| trainable | 999,885,952 / 999,885,952 (100%) |
| examples | 353,697 train, 12,374 held out |
| tokens per epoch | 310 M (mean 876 per example, measured) |
| maximum sequence length | 4,096 tokens |
| batch size | 16, gradient accumulation 4 |
| effective batch | 64 sequences |
| learning rate | 2e-5, cosine, 3% warmup |
| masking | prompt masked, loss on the answer only |
The user prompt is masked from the loss, so the model is trained to produce the assistant response rather than being rewarded for reproducing the question.
Quantization and local deployment
The final model is distributed as a GGUF checkpoint for llama.cpp. We selected Q4_K_M quantization because it provides a practical compromise between model quality, file size, memory consumption, CPU throughput, and deployment simplicity.
Our aim is not only to produce a model that fits into memory. We want enough remaining system memory for retrieval indexes, document processing, and the application layer.
Results
CPT — did the model learn the literature?
Measured on held-out papers the model never saw:
| metric | value | reading |
|---|---|---|
| domain perplexity | 5.94 | on African research writing |
| domain bits/byte | 0.7326 | tokenizer-independent |
| general perplexity | 11.56 | ordinary English |
| general bits/byte | 0.8064 | |
| span NLL (trained papers) | 4.8879 | retention |
| span NLL (held out) | 4.7255 | generalisation |
Domain perplexity is roughly half the general figure — the model finds African research writing substantially more predictable than ordinary English, which is the point of the exercise.
The African concept recall probe
A purpose-built probe: 400 cloze items over local concepts — Xylopia aethiopica, bambara groundnut, garri, bitter leaf, Irvingia gabonensis, African yam bean — measuring whether the model predicts them better in context after CPT.
| trained papers | held-out papers | |
|---|---|---|
| items / papers / distinct concepts | 200 / 137 / 89 | 200 / 129 / 92 |
| base span perplexity | 19.30 | 20.78 |
| CPT span perplexity | 12.84 | 14.78 |
| improvement | −33% | −29% |
| conditional win rate | 0.670 | 0.655 |
| first-token recall@5 (base → CPT) | 0.48 → 0.555 | 0.50 → 0.510 |
CPT wins on two thirds of individual concepts, and the improvement transfers to papers it never read — 29% on held-out, against 33% on trained. That gap is small, which is the encouraging part: this is domain learning, not memorisation.
An honest negative result
The probe also measures association gain — improvement on the true paper context minus improvement on a neutral context. If CPT taught genuine paper-specific association, this would be positive. It is not:
| trained | held out | |
|---|---|---|
| mean raw NLL gain | +0.4077 | +0.3408 |
| mean neutral NLL gain | +0.6879 | +0.6681 |
| mean association gain | −0.2802 | −0.3273 |
| 95% CI | [−0.391, −0.070] | [−0.463, −0.089] |
The model got better at these concepts generally — a stronger African-vocabulary prior — more than it learned which paper said what. The confidence intervals exclude zero, so this is a real effect, not noise.
We report it because it is the honest read of our own instrument, and it sets the agenda: rank, embedding-path coverage, and full-parameter CPT are the levers, and the SFT stage is where paper-specific association is most likely to be established.
SFT — and where we stopped
The full-parameter SFT run was halted deliberately at roughly step 1,547 of 5,527, because the held-out curve told us to:
| step | training loss | validation loss | |
|---|---|---|---|
| 250 | 0.7467 | 0.6037 | |
| 500 | 0.7279 | 0.5951 | |
| 750 | 0.7179 | 0.5951 | |
| 1000 | 0.7107 | 0.5930 | ← best |
| 1250 | 0.6850 | 0.6000 | rising |
| 1500 | 0.6537 | 0.6119 | rising |
The model converged at 18% of one epoch. Training loss kept falling while validation turned upward — the textbook signature of overfitting. The best checkpoint is step 1,000, and running to the end of the epoch would have produced a worse model at five times the cost.
This is the kind of finding that only appears if you wire held-out evaluation in from the start.
What it looks like in use
Closed book — recall, with a citation
Q. In the Ogun State rice-growing communities study, how often were respondents incapacitated by infection?
A. 45% of respondents were incapacitated 1–2 times, 27% 3–4 times and 19% were incapacitated more than 4 times.
Provenance: LEARNED_STUDYCitation: (Sam-Wobo et al., 2013)Study basis: discipline: PARASITOLOGY; population: consented volunteers (adults and school-age children) resident in the communities; period: May 2009 to March 2010
No evidence was supplied. The figures came from the weights, and the study is named precisely enough to verify.
Closed book — reasoning, with the inference marked
Q. What did the modified Gibbs diagram indicate about water composition in the Cross River Basin?
Reasoning. A plot on a modified Gibbs diagram places most of the water samples in the region of rock dominance, indicating rock weathering as a primary factor controlling the water composition.
Answer. The modified Gibbs plot places most samples in rock dominance, which the authors interpret as evidence for primary control by rock weathering.
Provenance: LEARNED_STUDYCitation: (Nganje et al., 2015)Study basis: discipline: HYDROLOGY_HYDROGEOLOGY; design: detailed geochemical study; population: surface water and groundwater sources; location: southeastern Nigeria; period: July–August 2009
Note the separation: the reasoning is stated, then the answer, then what the authors concluded — the model distinguishes a reported finding from its own inference.
Hedging when attribution is uncertain
Provenance: UNVERIFIED_STUDYCitation: (Sam-Wobo et al., 2013) [unverified]
The [unverified] marker is trained behaviour, not a post-hoc filter. The model learned to hedge when the attribution chain is weaker.
Where a general model gives you nothing useful
Questions MUFASA is built for and a general 1 B model is not:
- Which locally-available materials have been tested as partial cement replacements in Nigerian construction?
- What has been measured about helminth infection and agricultural labour productivity in southwest Nigeria?
- Has anyone characterised the hydrochemistry of shale bedrock aquifers in the Niger Delta?
- What gaps remain in African yam bean nutritional studies?
Running MUFASA
| requirement | |
|---|---|
| RAM | 7 GB (model ~800 MB + context) |
| GPU | not required |
| network | not required at any point |
| runtime | llama.cpp, Ollama, LM Studio |
Build llama.cpp
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j
Run MUFASA
./build/bin/llama-cli \
-m /path/to/MUFASA-Gemma3-1B-SFT-Q4_K_M.gguf \
-p "Explain how vegetation loss can reinforce drought conditions in the Sahel."
Once the model weights are downloaded, no cloud inference API is required.
This matters more than a benchmark number. A researcher on intermittent connectivity, on institutional hardware, working with unpublished data they cannot send to an API — this runs on their laptop, in a field station, on a plane.
Challenges we ran into
Defining "African research" without cheating
The naive filter — institution country — would have doubled our corpus overnight and made it meaningless. Writing a protocol that withholds affiliation from the classifier, and then accepting that it throws away 54% of candidates, was the hardest single decision in the project. It is also the one that gives the corpus its value.
Balancing quality and efficiency
Model size alone tells you very little about real deployment quality. Some larger models achieved stronger raw accuracy but required significantly more RAM. Other smaller models generated tokens quickly but produced weaker reasoning.
Model selection was never a single-metric problem. We had to weigh accuracy, throughput, latency, memory, and runtime reliability together.
Runtime compatibility
Different GGUF architectures do not always work with the same version of llama.cpp or llama-cpp-python. During benchmarking, a model could run through native llama-bench and then fail during accuracy evaluation because the installed Python binding was older than the runtime the architecture required.
We had to build the benchmark so that runtime versions could be isolated, one incompatible model would not stop the whole run, Hugging Face filenames resolved safely, large files were removed between runs, and memory was reclaimed before the next load.
Deployment engineering, it turns out, is part of model evaluation.
Preserving scientific documents correctly
Scientific papers are far longer than a 4,096-token context window, and the default training path truncates them. Measured, that would have discarded 28% of the corpus and the tails of 59% of papers — the discussion and conclusions.
Windowing correctly, and placing EOS only at true document boundaries, made the CPT corpus faithful to the original papers.
Extraction at scale is a resilience problem
Five gateways, thousands of papers, and JSON responses that arrive truncated, quote-mangled, or under keys the schema never specified. Every one of those failures, unhandled, costs a paper. Salvage, retry-and-keep-the-fullest, and per-record isolation were not polish — they were the difference between 10,131 processed papers and a much smaller number.
Training a small model effectively
A 1 B model has limited capacity, which makes data quality and stage design disproportionately important. Simply increasing sample count is not enough. We had to care about verification quality, duplicate control, clean held-out evaluation, response masking, document boundaries, optimizer configuration, domain coverage, and the distinction between domain learning and instruction learning.
Accomplishments that we're proud of
We wrote down what "African research" means, and enforced it. A published protocol, a labelled benchmark, and a 54% exclusion rate we can defend line by line.
We built a structured corpus, not a text dump. 10,480 papers became 479,143 QA pairs, 661,786 evidence spans and 265,833 observations across eight linked tables — with every claim anchored to a character offset in its source document.
We trained a model that knows when it is guessing. Provenance, citation, study basis and abstention are trained behaviours in the weights, not a wrapper around them.
We ran a reproducible model-selection benchmark instead of picking a base model because it was popular.
We trained a full-parameter 1 B scientific assistant — all 999,885,952 parameters updated, on 353,697 examples, after a dedicated domain-adaptation stage.
We produced a genuinely local model. GGUF Q4_K_M, roughly 800 MB, CPU-only, no retriever and no network at inference.
We kept deployment constraints visible throughout. MUFASA was not trained first and optimized later. Memory, CPU inference, quantization, latency and offline execution shaped the architecture from the beginning.
We published our negative result. The association-gain measurement says CPT built a vocabulary prior more than paper-specific association. We report it in full.
What we learned
The best model is not necessarily the model with the highest standalone accuracy.
A local AI system has to satisfy several requirements at once. Some larger models produced stronger accuracy but demanded substantially more memory; some compact models produced greater throughput but weaker reasoning. Gemma 3 1B gave us the balance worth specializing further.
Small-model training benefits from separating learning objectives. Continued pretraining teaches the statistical structure and vocabulary of the domain. Supervised fine-tuning teaches interaction. Keeping those stages distinct made the process far easier to reason about and evaluate.
Perplexity is a comparison tool, not a score. It is meaningful across checkpoints of the same model with the same tokenizer, and misleading across model families with different tokenizers. That is why we also report bits/byte and built a tokenizer-independent concept probe.
Measure your own instrument. The concept probe told us what we hoped to hear on span perplexity and something we did not want to hear on association gain. Both readings came from the same tool, and only one of them was flattering.
Held-out evaluation pays for itself immediately. Wiring validation in from the start is what let us stop SFT at 18% of an epoch instead of burning five times the compute to produce a worse model.
Model development does not stop when training ends. Runtime compatibility, quantization, memory management, file packaging, download reproducibility and offline execution all determine whether a model is actually deployable.
What we did not build, and why
A knowledge graph and retrieval layer. Designed, documented in 03-retrieval/, and deliberately not built. The challenge is a data-and-model-engineering competition, and the evaluation is of a raw GGUF answering prompts with no retriever attached. Given finite time we put it into the corpus, the extraction quality and the training data — the parts that determine what the model itself knows.
The architecture is worth reading as a statement of where this goes next: entity canonicalisation, licence-tiered retrieval, and a graph over the 663,518 entity mentions already extracted.
The african_innovation question family. The table exists with 9,935 rows and INNOVATION tags appear on 1,801 sampled examples, but a dedicated question family targeting local-material substitution was scoped and not implemented.
Full-epoch SFT. Stopped at 18% because the validation curve turned. That is a result, not an omission.
What's next for MUFASA
The next major step is to close the loop into a complete offline evidence-grounded scientific reasoning system. We do not expect a 1 B model to memorize the entire scientific literature. We want it to retrieve relevant evidence locally and spend its limited parameters on interpretation and reasoning.
The target architecture is:
User Question
|
v
Local Scientific Corpus
|
v
Evidence Retrieval
|
v
MUFASA 1B
|
v
Grounded Scientific Response
The model is already trained for the open-book half of this. What is missing is the retriever.
Our next steps:
- broader benchmarking of the final MUFASA SFT GGUF
- building the designed knowledge graph over 663,518 entity mentions
- integrating local, licence-tiered evidence retrieval with inline citations
- evaluating hallucination and citation fidelity directly
- raising the verified proportion of the SFT data
- implementing the
african_innovationquestion family - full-parameter or higher-rank CPT, to attack the association-gain result
- expanding African scientific literature coverage, especially in Technology and Materials
- domain-specific evaluation sets for agriculture, climate, geospatial and public health
- testing on affordable laptops and measuring real CPU energy consumption
- experimenting with additional quantization levels and knowledge distillation
What MUFASA stands for
| Letter | Meaning |
|---|---|
| M | Models |
| U | Understanding |
| F | Frontiers |
| A | African |
| S | Scientific |
| A | Advancement |
MUFASA = Models for Understanding the Frontiers of African Scientific Advancement
Vision
MUFASA is ultimately about making capable scientific AI more accessible.
Instead of assuming:
Large GPUs + Cloud APIs + Constant Connectivity
we are exploring:
Compact Models
+ Domain Adaptation
+ Local Evidence
+ Efficient CPU Inference
Our long-term goal is simple:
Build a scientific AI system that is small enough to run locally, specialized enough to be useful, grounded enough to inspect, and efficient enough to work on hardware people actually have.
Science that already happened, in a model small enough to carry.
Links
- MUFASA GGUF: huggingface.co/DestinyOtto/mufasa-gemma3-1b-sft-gguf
- Repository: github.com/Otto-Destiny/mufasa
- Africa Deep Tech Challenge 2026: adtc-2026.devpost.com
Repository code and documentation are released under the Apache License 2.0.
Built With
- cpt
- gemma
- huggingface
- llm
- python
- sft
Log in or sign up for Devpost to join the conversation.