-
-
“AI-powered learning, built to work offline making education more accessible for every learner, anywhere.”
SMART TUTOR
Inspiration
Twenty-seven percent. That's how much of Sub-Saharan Africa was using mobile internet by the end of 2023, according to GSMA — against 60% who live inside network coverage but still don't use it. The gap isn't coverage. It's affordability, devices, and skills. A tutor that needs a round trip to a server doesn't help the other 33%.
Then there's a number that doesn't get talked about nearly enough: UNICEF estimates close to 29 million children with disabilities live in Eastern and Southern Africa. In Kenya specifically, the 2023 Support Needs Assessment found that 55% of children with disabilities weren't in school at all — not underserved, not struggling, absent. Lack of special schools, inaccessible buildings, no assistive devices, and cost, in that rough order.
Put those two numbers next to each other and the question stops being "how do we bring AI to education" and becomes something narrower and harder: can a tutor exist that doesn't assume broadband, doesn't assume a screen a sighted user can navigate with a mouse, and doesn't assume a laptop bought this decade?
We didn't want to build a chatbot with a curriculum bolted on. We wanted to find out what an AI tutor looks like when every one of those assumptions is removed at the start, not patched in afterward. That's SMART TUTOR.
What it does
SMART TUTOR is a mathematics and science tutor that runs entirely on a CPU laptop with no internet connection, ever — not "works offline as a fallback," but architecturally incapable of making a network call while answering a question. Ask it something like:
A tank holds 240 litres. A pump fills it at 8 L/min while a leak drains 3 L/min. It starts one quarter full. How long until it's full?
and it walks through the algebra step by step, checks its own arithmetic, and closes with a single machine-checkable line — FINAL ANSWER: 36 — so a learner, a teacher, or an automated grader never has to hunt through prose to find the number that matters.
Accessibility isn't a display theme sitting on top of that. It's a rendering layer that takes the same reasoning and produces four different outputs from it: plain terminal text; a linear screen-reader form with no visual decoration to trip a reader up; large-print with short wrapped lines and blank space between steps; and a spoken form, where 3x + 7 = 22 is rewritten as "three x plus seven equals twenty-two" before it ever reaches a TTS engine, because screen readers choke on notation the way a human wouldn't. Speech output runs through pyttsx3 against the operating system's own voices — no network, no API key.
Underneath, it's Qwen2.5-Math-1.5B-Instruct, QLoRA fine-tuned and quantized to a 940 MB GGUF file, driven by llama.cpp on CPU only. No GPU, no Docker, no API key. Total resident memory at generation time: about 1.7 GB, against an 8 GB budget. On the development machine it answers at roughly 17 tokens a second — faster than most people read out loud — with 96.7% held-out accuracy (29/30) on a math and science set spanning ten categories.
How we built it
The pipeline is short on purpose: prompt → llama.cpp → GGUF weights → answer. Every optimization decision had to earn its place against that path, not against a paper claim.
The model. Training happened where the compute was — a free Colab T4. QLoRA touched 2.34% of parameters (37M of 1.58B) across the attention and MLP projections, 150 steps, loss computed only on the assistant's turn so the model doesn't learn to generate the question back at you. Then the adapter merges into dense fp16, gets converted to GGUF, and quantized to Q4_K_M — the only step that actually determines inference speed. QLoRA makes training cheap; it does nothing for decode speed, and we didn't want a report that implied otherwise.
The offline guarantee. We also built something most hackathon submissions skip: a way to prove the offline claim rather than assert it. Three layers — a static AST audit of the inference path that would catch a network import even hidden inside a docstring-adjacent comment, a runtime guard that replaces socket.socket so any outbound call raises immediately, and an end-to-end test that generates a real answer with that guard live, including inside a Linux network namespace with literally no route to anywhere.
The research layer. Alongside the tutor we built UATL — the Unified Adaptive Transformer Layer — a proposed policy coordinating numerical precision, uncertainty estimation, and KV-cache compression per token, at every transformer layer \( l \):
$$h_{l+1} = h_l + G(d_l)\,\big[F(Q_{p_l}(W_l),\, T_c(X_l),\, K_q,\, V_q)\big]$$
where difficulty \( d \) is the entropy of the next-token distribution, in nats:
$$d = -\sum_i p_i \ln p_i$$
\( Q_{p_l} \) schedules per-layer precision \( p_l \in {2,3,4,8} \) bits, \( T_c \) is cache-aware tiling, \( K_q, V_q \) are quantized KV-cache reads, and \( G \) is an execution policy realising the difficulty signal as work actually removed — token budget, early exit, routing — rather than a hidden multiplier sitting on top of a matmul that still runs at full cost. The whole system is a constrained optimisation:
$$\min T_{\text{token}} \quad \text{s.t.} \quad A \ge A_0 - \epsilon,\;\; M_{\text{peak}} \le 7\text{ GB},\;\; \Theta_{\max} \le 85^{\circ}\text{C}$$
None of the individual mechanisms are new, and we say so plainly rather than presenting established ideas as inventions: entropy-gated compute is CALM (Schuster et al., NeurIPS 2022); layer-wise mixed precision is a mature quantisation-literature technique. What UATL proposed differently was switching precision per token, at runtime, on a frozen GGUF running through unmodified llama.cpp — and testing that difference against real hardware turned into the project's most important finding.
The one rule that mattered. A technique doesn't ship until PrecisionCostModel, populated by real benchmarks, shows it faster on this CPU — enforced in code, not asserted in a citation. Applied to the KV cache, exact arithmetic first corrected our own design notes by 8×:
$$M_{kv} = 2\,L\,H_{kv}\,d_{\text{head}}\,T\,b = 2 \times 28 \times 2 \times 128 \times 2 = 28{,}672\text{ bytes/token}$$
not the 224 KB/token a multi-head assumption would give — Qwen2.5's grouped-query attention (2 KV heads) makes the cache small to begin with. We quantized it anyway, to see what the measurement-gated rule would say, and it measured 11% slower with zero memory saved. Disabled. The same rule rejected a narrower quantisation format for the adaptive-precision path — smaller on paper, measurably slower on this CPU, because llama.cpp's low-bit kernels dequantise to a wider type before multiplying, so the unpacking cost exceeds the bandwidth saved. (This is exactly the gap table-lookup kernels like T-MAC exist to close — our negative result and that line of research are the same observation from opposite ends.) Three separate times, the measurement-gated rule changed what shipped. Zero times did a plausible-sounding optimisation get in without proving itself first.
Challenges we ran into
The model that scored perfectly on our own held-out math set still failed the product's actual contract: it answered \boxed{5} instead of the FINAL ANSWER: 5 line every downstream system — the grader, the CLI, the accessibility renderer — depends on. Getting math right and getting the output format right turned out to be two separate problems, and only one of them shows up if you just eyeball whether the answer is correct.
Fixing that with fine-tuning surfaced a second, quieter bug. On a held-out multi-step problem, the model derived the right answer mid-reasoning, then correctly verified it — and then its FINAL ANSWER line reported the verification number instead of the answer to the actual question. On another, it stated out loud that a result needed to be rounded up, and then reported the raw unrounded decimal anyway. We caught both by testing against the literal wording of our own two showcase prompts, not synthetic ones. A one-line addition to the system prompt — restate the value the question asked for, don't substitute a verification number, apply the rounding you just said you'd apply — fixed both, verified by rerunning the exact prompts before and after, and confirmed at the dataset level: held-out accuracy moved from 27/30 to 29/30.
One failure remains, and we're naming it rather than hiding it. A tersely-phrased variant of the tank problem — no "write an equation and verify" scaffolding — still divides the wrong quantity, even though the scaffolded showcase-prompt wording of the identical problem is answered correctly. That's a genuine reasoning gap, not a formatting one, and we didn't patch the prompt to specifically fit that one held-out item. Doing so would have fixed the benchmark, not the model.
Then there was a run of small platform papercuts that each looked, at first, like "the model is broken": a stale system C++ runtime causing the benchmark binary to crash with an access violation; a Windows path-length limit that silently killed a dependency's install partway through unpacking; a Unicode character in a third-party tool's own progress output crashing the whole profiler run before it produced a single number. None of these were our model failing. Every one of them would have read as "doesn't work" if we hadn't dug into the actual crash instead of the surface symptom.
And the deepest challenge was realizing partway through that "accessible" isn't one target. A blind learner, a low-vision learner, and a learner who needs shorter sessions because of load-shedding all need the same reasoning rendered three different ways — which meant the accessibility layer had to sit downstream of a single source of truth, never duplicate the tutoring logic, and never touch the benchmarked path at all.
Accomplishments that we're proud of
On our held-out set — 30 problems across ten categories, with an answer key we independently re-verified rather than trusted — the fine-tuned model scores 29/30 (96.7%), with a FINAL ANSWER: line in the exact parseable format on every single item, including the one graded wrong. Nine of ten categories are a clean 1.000; the one open gap is documented above, not smoothed over.
We're proud that our negative results sit next to the positive ones, with equal weight. Grouped-query attention means our KV cache costs 28 KB per token, not the 224 KB a naive estimate would suggest — that's real. Quantizing that KV cache to save more memory measured 11% slower, not faster, so it's disabled in the shipped config, and we said so instead of burying it. Python-level cache tiling lost to the plain matmul call at every tile size we tried. A benchmark that only reports wins isn't a benchmark; it's marketing.
We're proud of the offline guarantee having three independent layers instead of one comment saying "this works offline." And we're proud that on this development machine — an ordinary laptop, not the target hardware — throughput alone already saturates the performance score, at 17.1 tokens/second against a 15 tok/s reference.
What we learned
The biggest lesson had nothing to do with model architecture: the largest model is not automatically the best model for this problem. A 1.5B model that loads in about a second, fits in 1.7 GB, and runs on hardware a school already owns is more useful here than a much larger model that requires infrastructure nobody in the target environment has.
We learned that a "correct" model and a usable one are different claims, and the gap between them lives in unglamorous places — an output marker, a rounding step, a stale system DLL — not in the headline capability. We learned that testing against your own showcase examples, in their exact wording, finds bugs that synthetic test sets miss, because judges and real users don't paraphrase your test cases; they run the ones you gave them.
We learned that our own optimisation intuitions were wrong in the same direction, every time — all optimistic. "Fewer bits is faster" felt obvious and measured false. "Compressing the KV cache saves memory" felt obvious and measured false too. The instinct to trust a roofline argument over a real measurement is exactly the instinct this project was built to distrust.
And we learned, again, that accessibility has to be a starting constraint, not a feature added once the "real" product is done. Every accessible rendering mode in this project reuses one reasoning path. That was only possible because we designed the seam before we wrote the CLI, not after.
What's next for SMART TUTOR
The honest next step is testing with the people this is actually for — sessions with NVDA and Orca screen-reader users, and with teachers of blind learners, because we do not want to keep guessing at what "accessible" means from the outside.
Closing the one remaining reasoning gap is next in line: the terse-phrasing failure above likely needs more varied phrasing in the fine-tuning data, not another prompt patch — the fixes that were fixable that way are already shipped.
Sign-language interaction is on the roadmap, and we're naming it as unsolved rather than pretending otherwise: there is no open sign-language model — Kenyan, Ugandan, or South African — of the quality a maths tutor would need without competing for the same memory budget as the tutor itself. The architecture already takes a plain question string as input, so a recognizer has a seam to attach to the moment one exists. The same is true for speech-to-text: deferred on purpose, because an offline STT model that's actually good enough would fight the tutor for the 8 GB budget, and that trade needs to be measured before it's shipped, not assumed.
Beyond that: Kiswahili support, because instruction in a learner's first language measurably improves comprehension and we don't yet have evaluation data to build that responsibly; batch tooling for teachers who need worked solutions for an entire problem set; and finishing the adaptive-precision research thread properly with multiple quantisation artifacts to route between, so the remaining ablation cells can run instead of being correctly skipped.
None of this replaces a teacher. It's a tool for the learner and the teacher who don't currently have one that fits in their bag, runs without a connection, and reads its own answers out loud. That's the part of the AI story that's been missing, and it's the part we set out to build.
Log in or sign up for Devpost to join the conversation.