Inspiration
Qemer began with a simple access problem: programming help is often most needed where dependable internet access, paid AI APIs, and current online documentation cannot be assumed. I wanted to explore an offline coding assistant that can explain code, support debugging, and eventually ground its answers in local technical documentation.
What it does
Qemer is an offline coding-assistant project. This submission packages and profiles its local model layer: Qwen3.5-0.8B in GGUF Q4_K_M format, running through llama.cpp on CPU.
The broader Qemer project is designed to retrieve relevant documentation locally, then use those snippets to ground coding explanations and answers. That companion retrieval application is being developed separately; this submission focuses honestly on the model component measured by the ADTC profiler.
How I built it
I evaluated small local models and quantizations against the practical constraints of CPU-only inference, memory use, and response speed. I selected Qwen3.5-0.8B with Q4_K_M quantization because it provides a compact GGUF artifact with substantial memory headroom for the ADTC laptop profile.
The submission uses a reproducible download script, structured metadata, and the ADTC profiler with llama.cpp. I am developing Qemer as an open-source Rust project with separate retrieval, answer-generation, and terminal-interface layers.
Challenges I ran into
The main challenge was balancing useful coding behavior against the limits of an 8 GB laptop. Larger models can improve general capability, but they cost more memory and reduce CPU throughput.
I also learned that benchmarks from a development laptop can be misleading: CPU instruction sets, thread count, memory capacity, and container settings materially affect results. I therefore focused on reproducible CPU-only profiling and kept the boundary between the measured model layer and the in-progress retrieval application explicit.
Accomplishments that I'm proud of
- Produced a reproducible, public submission package for a local GGUF model.
- Selected a model small enough to leave substantial memory headroom on the target profile.
- Verified the model metadata against the GGUF header and ADTC profiler schema.
- Designed Qemer around offline documentation grounding rather than relying only on model memory for version-specific programming details.
- Kept the project open source and transparent about its current implementation scope.
What I learned
I learned that local AI performance is a systems problem, not just a model-selection problem. Quantization, CPU behavior, memory usage, packaging, and reproducibility all directly affect whether a model is usable on ordinary hardware.
I also learned that an offline coding assistant needs trustworthy context. A small model becomes more useful when it can retrieve the right local documentation instead of guessing API details from its parameters alone.
What's next for Qemer
My ambition for Qemer is bigger than a local chatbot: I want to build a trustworthy, offline knowledge layer for developers, students, and small engineering teams across Africa.
The next step is to complete the documentation-retrieval pipeline and turn Qemer into a polished local coding environment that can search versioned technical references, show its sources, and generate grounded explanations without sending a developer’s code or questions to the cloud.
Beyond the terminal interface, I want Qemer to support reusable local documentation packs, editor and agent integrations, and African developer communities that need reliable tools despite limited connectivity. The long-term goal is simple: useful programming intelligence should not depend on expensive APIs, permanent internet access, or foreign infrastructure.
Log in or sign up for Devpost to join the conversation.