• 

  Offline Coding Assistant

Offline Coding Assistant for African Laptops

The Problem

AI coding assistants are increasingly useful, but many developers and students in Africa work with modest hardware, unreliable connectivity, limited data budgets, and no discrete GPU.

We explored a practical alternative: can useful coding assistance run locally, privately, and continuously on hardware people already have?

What We Built

We built and benchmarked an offline coding-assistant configuration using SmolLM2-135M-Instruct, quantized as GGUF Q4_K_M, and executed through llama.cpp.

The model and runtime are designed for CPU-only local inference. During inference, prompts and source code do not need to be sent to a cloud API.

The project targets the ADTC 2026 Coding Assistants domain, with programming generation and tutoring as the primary use case.

Official Participant-Laptop Benchmark

The final participant run recorded in submission.json measured:

• Model: SmolLM2-135M-Instruct-Q4_K_M
• Parameters: 134.52M
• Model size: approximately 98.87 MiB
• Runtime: llama.cpp
• Participant environment: ARM64/aarch64 Linux environment with 7.5 GB RAM and no GPU
• Generation throughput: 25.58 tokens/second
• First-token latency: 8.94 seconds
• Peak RSS: 175.12 MB
• Steady-state RSS: 110.35 MB
• Thermal throttling: false

The official participant benchmark remains the primary measurement for this submission. It is not replaced by the mobile validation below.

Supplementary ARM64 Mobile Validation

We additionally validated the same model and GGUF runtime on an ARM64 Android device using Termux and an ARM64-native llama.cpp build.

These results are supplementary and are not presented as the ADTC participant-laptop benchmark.

Metric	Prompt 1	Prompt 2	Average
Generation speed (tokens/s)	32.86	32.67	32.77
Prompt evaluation (tokens/s)	99.94	101.63	100.79
Total execution time (s)	1.492	1.486	1.489

Mobile environment:

• ARM64 / aarch64
• Android
• Termux
• ARM64-native llama.cpp
• CPU-only execution

This demonstrates that the same compact model/runtime can also operate on constrained ARM64 mobile hardware, extending the accessibility story beyond the laptop target.

Why This Matters

The key result is not simply that a small model can run locally. It is that useful AI assistance can be packaged, measured, and reproduced around constrained hardware without requiring cloud inference.

For students, developers, schools, and small organizations operating under connectivity, cost, privacy, or hardware constraints, local inference provides another path to practical AI assistance.

The mobile validation also suggests a useful deployment direction: when an 8GB laptop is unavailable, the same compact inference stack can be tested on capable ARM64 phones.

What We Learned

The work highlights the trade-offs involved in constrained AI engineering. Model size, quantization, memory footprint, runtime choice, CPU utilization, latency, throughput, and workload quality must be considered together.

It also reinforced the importance of reproducible measurement. Rather than estimating whether local AI is "fast enough," we used the ADTC profiler for the participant run and recorded separate runtime measurements for ARM64 mobile validation.

What's Next

The current configuration establishes a reproducible foundation for an offline coding assistant. The next stage is to improve the practical assistant layer around the model while preserving the core constraint of local, low-resource inference.

Potential improvements include better prompt handling, code-context management, retrieval of local project files, lightweight editor integration, and further CPU-specific optimization.

The broader goal is simple:

make useful AI assistance possible on the hardware people already have.

Repository

The complete open-source submission, benchmark artifacts, model download script, metadata, and technical report are available in the project repository.

Built With

• c++
• gguf
• llama.cpp
• python
• smollm2
• termux