Setlhare 🌳 — the offline pair programmer

Setlhare (Setswana for tree, as in solution tree) is an on-device coding assistant that lives in your terminal: run any command through it, and when it fails, Setlhare reads the stack trace, finds the broken code, and asks a local LLM for a diagnosis, a Git diff patch, and an explanation — 100% offline.

Inspiration

Every developer knows this loop: hit an error, alt-tab to Stack Overflow or an AI chat, paste the traceback, wait on slow internet, lose your flow. In Lagos, Arusha, or Dakar, that loop often breaks entirely — fiber is expensive, electricity is unreliable, and API fees are real money. I kept thinking: the error is already on my machine. The code is already on my machine. Why isn't the help? So we put everything in the terminal: you never have to leave.

What it does

python cli.py fix "python3 buggy.py" runs your command like normal. If it crashes, Setlhare:

1. Parses the stack trace — Python tracebacks, Java exceptions (NullPointerException included), and JavaScript/Node errors
2. Extracts context — ±15 lines around the failing line plus the enclosing function via AST
3. Asks a local model — Qwen2.5-Coder-1.5B (Q4_K_M GGUF) via llama.cpp at temperature 0.1, fully offline
4. Prints a diagnosis, unified diff patch, and plain-language explanation

How we built it

• Model selection was measured, not guessed. We benchmarked three models head-to-head on a real repair task on hardware weaker than the challenge's reference laptop (i5-6200U, 5.8 GB RAM): the 3B produced great text but ran at 98.9% CPU with 3.4 GB RSS; our own fine-tune formatted beautifully but its patches were wrong; the base 1.5B was fast AND correct. We shipped the 1.5B.
• A fine-tune that taught us more by failing. We LoRA-tuned Qwen2.5-Coder with Unsloth on 4,000 commit-fix examples + 1,500 synthetic error→repair samples with verified working diffs. It learned the format perfectly — and regressed on correctness. That negative result is documented in REPORT.md; knowing when not to ship your fine-tune was the most valuable engineering call of the project.
• Systems hardening. Multi-language parser with unit tests, resumable credential-free downloads validated by GGUF magic bytes, bounded inference timeouts, and graceful degradation everywhere. We simulated the evaluator with a fresh git clone before submitting.
• Validated with the official profiler: adtc-profiler confirms ~6 tokens/s generation, ~1.8 GB peak RAM, no thermal throttling.

Challenges we ran into

• A newer llama.cpp build silently ignored -no-cnv and hung forever in conversation mode — the exact kind of bug that fails evaluations silently. Found it by running the real pipeline end-to-end, fixed with --no-conversation --single-turn.
• Commit-message diffs turned out to be poor training targets for runtime error repair — models learn "make a plausible diff", not "fix the bug".
• 1.1 GB of weights briefly ended up committed to git; caught and removed before submission.

What we learned

Small models with tight, structured prompting beat big models on constrained hardware when the task is narrow. Measure before you fine-tune. And the hardest bugs are the ones where the tool appears to work.

What's next

Streaming output, a tutoring-mode subcommand for students (setlhare explain), multi-file context retrieval, and a larger curated runtime-error corpus to take another shot at the fine-tune.

Run it yourself

git clone https://github.com/nyaks1/setlhare && cd setlhare
bash download_model.sh        # ~1.1 GB, idempotent, no credentials
python cli.py fix "python3 buggy_script.py"
Full technical write-up, benchmarks, and design decisions in
REPORT.md (https://github.com/nyaks1/setlhare/blob/main/REPORT.md).

Built With

• bash
• gguf
• huggingface
• llama-cpp
• lora
• python
• qwen2.5-coder
• ubuntu
• unsloth