Baarali Edge

An offline enterprise assistant for refurbished 8 GB laptops: it reads your company's own documents, answers with citations, and never sends a byte to the cloud.

Comment

Inspiration

A laptop has a battery. A fibre box does not. In Côte d'Ivoire, 80.4% of firms report power cuts, and the cloud assistant goes down at exactly the moment the work becomes urgent — a quote to send, a contract clause to check, a report due.

There is a second reason, and it is the one companies say out loud: supplier contracts, payroll and client files cannot be sent to a cloud provider at all. Not for cost, for confidentiality. An assistant that requires uploading those documents is not a cheaper option. It is not an option.

So we built for the machine those companies already own: 8 GB of RAM, integrated graphics, no network.

What it does

Baarali Edge is an offline enterprise assistant for the corporate_enterprise domain — knowledge-work productivity for small and medium businesses: summarising, drafting and analysing a company's own documents, on the machine, in French and English, with no network at any point.

The same reasoning core is reachable over USSD from a feature phone, at the real 182-character screen limit, with no data plan and no app.

How we built it

By measuring, not by choosing. Every decision below was made with the official adtc-profiler on an 8 GB laptop, CPU only (-ngl 0), median of three runs.

  • Base model. Five open models profiled, from 0.75 B to 4.21 B measured parameters. Qwen3.5-2B won on total score, not on accuracy.
  • Quantisation. All seven quantisations of the winner, profiled end to end. IQ4_XS won: 31.20 tokens/s, 1 544 MB peak RSS against the contest's 7 GB ceiling, giving S_eff 77.9, and 0.67 on 200 arc_easy questions.
  • Sampling. repeat_penalty = 1.05, because llama.cpp applies none by default and this model loops on inputs outside its competence.
  • Verification. Offline proven twice: no non-local socket across 15 samples during profiling, and a full inference with the Wi-Fi physically off. The weights are fetched by download_model.sh with a SHA-256 check.
  • Calibration. We rebuilt the importance matrix on our own enterprise corpus and measured it against a purpose-built control. It changed nothing: 196 of 200 answers identical. The negative result is in the report, with the protocol that makes it credible.

Challenges we ran into

We published a decision, then had to reverse it. An 18-item arithmetic control pointed at repeat_penalty = 1.10. It was rigorous, documented — and off-topic, because our domain is summarisation, drafting and analysis, not arithmetic. So we built a second control on the official domain definition: 15 tasks, French and English, scored without human judgement, with a detector for invented numbers. It showed that at 1.10 the model answers our own published test prompt by fabricating a formula. We reversed to 1.05 and wrote down why. A measured number is only worth as much as the question you asked it.

The hidden validation set is not distributed to participants. The profiler source says the judges hold it. There is nothing to train against, so the only defence against the two hidden prompts is breadth — which is what those 15 tasks are for.

Accomplishments that we're proud of

Not a score — a discipline. No number enters our report unless it comes out of a measurement, and the report contains the ones that do not flatter us: the quantisation that beats ours on accuracy is in the table, with the arithmetic showing why it still loses. The 4 B model that is 6 accuracy points better is there too, with the reason it loses on total score.

We also removed a claim rather than defend it. We had declared Dioula in our language scope. We probed it: the model answered that Dioula is "the language of Cameroon". We deleted dyu from the submission.

What we learned

That half of this contest is won by not being big. Throughput and memory carry 50% of the score, throughput is graded relative to the fastest submission, and the accuracy a larger model buys is rarely worth what it costs in both.

And that documentation is scored, not decorative — it sits inside the 50%.

What's next for Baarali Edge

We had identified one lever and left it unpulled: the importance matrix, whose calibration our file inherits — generic English, 80 chunks, computed by someone else. We have since pulled it, and it does not move.

We built a 176 kB calibration corpus of enterprise documents, 55 % French, its length chosen by measuring 4-gram diversity rather than by preference. A contamination check found an eight-word overlap between one of our templates and one of our own control tasks — written by the same hand — and we rewrote the template before computing anything: a corpus containing its own exam marks its own paper. The rebuild then had to be compared against the right thing, which took two more files: a control on the inherited calibration, and a candidate replaying the shipped file's tensor map exactly, landing 32 bytes from it with an identical peak RSS.

Throughput and memory: unchanged. Accuracy: 0.670 against 0.680 on 200 questions — and because two totals four questions apart prove nothing, we compared them question by question. 196 answers out of 200 are identical. p = 0.625. We kept the shipped weights and published the negative result.

That leaves the next lever a real one rather than a guess: a format more aggressive than IQ4_XS, where fewer bits remain to allocate and more is at stake in allocating them well.

Built With

  • bash
  • cpu-inference
  • edge-ai
  • gguf
  • hugging-face
  • imatrix
  • iq4-xs
  • llama-cpp-python
  • llama.cpp
  • lm-evaluation-harness
  • offline-first
  • on-device-ai
  • python
  • quantization
  • qwen3.5
  • ussd
Share this project:

Updates