Africa Deep Tech Agent

Offline multi-agent diagnostics for 8 GB laptops: llama.cpp + GGUF, local RAG, and human approval before any action. Peak RAM ~2.1 GB. No cloud at inference.

Comment

Inspiration

Cloud copilots fail where connectivity is expensive or unstable, and they don't fit an 8 GB laptop. Africa Deep Tech Challenge 2026 asked for useful on-device AI on hardware people already own. I wanted an agent that diagnoses local services and Docker problems offline, with a human still in control.

What it does

Africa Deep Tech Agent is an offline multi-agent diagnostic system:

  1. Triage — classifies intent, severity, and route
  2. Local RAG — retrieves runbooks and (if connected) your project files
  3. Diagnostic — forms root-cause hypotheses with evidence
  4. Resolution — proposes a structured action plan with risk labels
  5. HITL — Dry-run, Approve, or Reject. Nothing mutates the machine by default

A Tauri desktop app starts a local Python API. You can bind a real project folder and run allowlisted read-only Docker/file checks.

How we built it

  • One shared GGUF (Qwen2.5-1.5B-Instruct Q4_K_M) via llama.cpp — three agent roles, not three models
  • Python orchestrator, RAG (NumPy + hash embeddings), allowlisted executor
  • Tauri (Rust) window that starts src/server.py in the background
  • HITL gate: medium/high-risk steps never auto-execute
  • Official ADTC files: metadata.json, download_model.sh, REPORT.md

Peak RSS ~2.1 GB. Generation ~63 tok/s on a participant laptop smoke test.

Challenges we ran into

  • Small models break JSON — we added schema prompts, parse/repair, and heuristic fallbacks
  • Port 8000 was already used by another Docker app — the desktop API now uses 8765
  • Finder-launched apps don't see Docker on PATH — we inject Homebrew/Docker paths
  • Real exec had to stay safe: allowlist + dry-run default + no restarts without approval

Accomplishments that we're proud of

  • Fully offline inference, llama.cpp only
  • Multi-agent pipeline that is more than a chatbot wrapper
  • Live inspect against a real Compose stack (docker ps, compose config)
  • 35 unit tests; RAM well under the 7 GB budget
  • One-click desktop icon for the demo

What we learned

Constraint-first design beats a bigger model. Shared weights, short structured outputs, and a hard HITL gate make an agent usable on an 8 GB machine.

What's next for Africa Deep Tech Agent

  • Full ADTC profiler run with accuracy (not only --skip-accuracy)
  • Optional 3B profile if RAM allows
  • More domain runbooks and clearer live-evidence summaries

Built With

Share this project:

Updates