Mentat-Lite

An on-device, zero-network AI study studio & exam-prep tutor for Nigerian students. Delivers grammar-enforced STEM lessons, diagrams, and quizzes on 8GB laptops at ₦0 cloud cost.

Comment

Inspiration

Over 2 million Nigerian students sit for high-stakes national examinations (UTME/JAMB, WAEC, NECO) every single year. For most of these candidates, studying with modern AI tools is completely out of reach. Cloud-hosted LLMs demand recurring API subscriptions, stable fiber internet, and continuous electricity. During power blackouts in Lagos, Kano, or Ibadan, or in off-grid rural schools with zero cellular connectivity, cloud AI simply does not exist.

We asked a fundamental systems question: What if the AI tutor ran 100% on the laptop the student already owns, with zero internet and zero token fees?

That question gave birth to Mentat-Lite: an autonomous, on-device AI tutor and interactive STEM Study Studio engineered to deliver sub-second, deterministic exam preparation on standard 8 GB commodity laptops.

What It Does

Mentat-Lite transforms an entry-level 8 GB laptop into a zero-data, air-gapped exam prep station:

  • Interactive Guided Lessons: Structured, step-by-step concept explanations featuring active-recall checkpoints and locally rendered Mermaid.js SVG diagrams with zero external network requests.
  • Grammar-Enforced MCQ Practice: Generates authentic WAEC/JAMB exam practice questions with instant grading and step-by-step mathematical reasoning.
  • Spaced-Repetition Flashcards: Offline flashcard engine driven by an SM-2-lite spaced-repetition algorithm to maximize long-term retention.
  • Offline Textbook OCR Scanner: Scans printed textbook and past-question pages via webcam using bundled local Tesseract WASM binaries.
  • Authentic African Grounding & Multilingual Support: Communicates in English, Nigerian Pidgin (pcm), Yoruba (yo), Hausa (ha), and Igbo (ig), grounded in a local corpus of real WAEC and JAMB past exam questions.
  • Local Mastery Graph: Continuously calculates per-topic mastery scores in local storage, dynamically steering the tutor's system prompt toward the student's weakest subjects.

How We Built It

  • Base Model & Quantization: We selected Llama-3.2-3B-Instruct and quantized it to GGUF Q4_K_M (~2.0 GB on disk). This represents the sweet spot for 3B-class models: halving memory versus Q8 with negligible quality loss on multi-step reasoning.
  • Desktop Runtime: Wrapped node-llama-cpp inside a high-performance Electron + React + TypeScript desktop shell.
  • Grammar-Constrained Decoding: Implemented strict schema enforcement via llama.createGrammarForJsonSchema(). The 3B model is mathematically constrained by context-free grammars to emit valid JSON structures for lessons, quizzes, and diagrams—completely eliminating malformed parsing errors.
  • Zero-Network Architecture: Stripped all external webfonts (using native OS font stacks) and bundled all diagramming and OCR assets locally to ensure 100% offline compliance.
  • Local LAN Distribution: The host machine can act as a local LAN hub, allowing 50+ student devices in a disconnected computer lab to connect concurrently via local Wi-Fi.

Challenges We Ran Into

  • The 8 GB RAM Ceiling & KV-Cache Bloat: Local LLMs easily exhaust RAM as context grows. We hard-capped the sequence context at n_ctx = 4096 in electron/main.js, keeping peak resident memory strictly at ~3.2 GB—leaving a massive safety buffer under the 7.0 GB challenge limit.
  • Electron IPC Sequence Exhaustion: Rapid multi-turn queries in node-llama-cpp initially triggered sequence creation leaks. We re-architected the backend to reuse persistent LlamaChatSession instances and lazily instantiate isolated structured contexts.
  • Deterministic Visual Generation: Small models often hallucinate broken formatting. By forcing Mermaid syntax into strict JSON schema fields and building a resilient client-side sanitizer, our renderer deterministically paints complex STEM flowcharts entirely on-device.

Accomplishments That We're Proud Of

  • Measured Leaderboard Metrics:
    • Throughput (S_perf): 17.17 – 32.11 tok/s on CPU (well above the 15.0 tok/s reference → 100/100 Max Performance Score).
    • Memory Efficiency (S_eff): Peak RSS of 3272.6 MB (~53.0 efficiency score with zero OOM crash risk).
    • Accuracy (S_acc): 0.72 (acc_norm) on ARC-Easy / STEM reasoning benchmarks.
    • Thermals (P_thermal): 0 penalty (zero thermal throttling under sustained load).
  • 100% Air-Gapped Operation: Zero network packets leave the machine during inference.
  • Full Product Experience: Delivered a full-featured desktop Study Studio rather than a bare terminal script.

What We Learned

  • Small Models with Grammar > Large Models with Unconstrained Text: A 3B model constrained by strict JSON grammars is far more reliable and usable for structured workflows than an unconstrained 14B model that randomly emits markdown formatting bugs.
  • Access Economics Decide Real-World Impact: The future of AI in emerging markets is not about larger cloud data centers; it is about extreme algorithmic compression on the commodity 8 GB laptops that Africans already own.

What's Next for Mentat-Lite

  • Fine-Tuning on African Exam Reasoning: Training a custom Mentat-Math-1.5B on our 20-year digitized WAEC/JAMB dataset using Unsloth GRPO / SFT.
  • B2B School Lab Deployment: Packaging Mentat-Lite onto local master PCs and Raspberry Pi edge servers for private secondary schools and CBT centers across Nigeria.
  • Multi-Sector Expansion: Porting the core air-gapped Edge OS to healthcare clinics (WHO clinical triage) and power utilities (substation switching SOPs).

Built With

Share this project:

Updates

Submission history