Doculo

Ask your own invoices and contracts. Cited answers, fully offline - on an 8 GB laptop, no GPU, no subscription, no internet.
Same incident, different document: the lease puts the roof on the landlord. Two answers, two sources, one dispute worth having.
In the profiler's own 4 CPU / 8 GB container: 30.73 tokens/sec, 1.07 GB peak RSS, arc_easy 0.723 at n=300, 18/18 grounded QA.
There is no Nairobi branch. Nothing clears the similarity floor, so the model is never called. It declines instead of inventing.
The policy excludes water from a roof leak unless reported in 14 days. A claim that would have been refused - cited, in 5.4 seconds.
Seventeen checks and 18/18 on the QA set, run from a bare clone. Throughput reproduced within 9%, peak memory within 0.02%.
Retrieval and generation both run through llama.cpp as subprocesses. No PyTorch, no vector database, nothing compiles at install.
A price-revision notice dropped in mid-session: embedded locally, answerable seconds later, cited to the new file. Nothing uploaded.
Ten documents from a Douala trader, deliberately awkward: the supplier is named differently on the contract and on the invoice.

Inspiration

A small trading business in Douala accumulates the same paperwork as a large one: supply agreements, invoices, a warehouse lease, notes scribbled after customer visits. What it doesn't have is anyone whose job is to remember what's in them. When a supplier disputes a payment term, the answer exists - spread across a folder of PDFs, a WhatsApp thread, and an email account.

The tools that would answer that question assume a fast connection, a subscription in hard currency, and RAM to spare. There's a fourth assumption nobody mentions: willingness to upload your scanned contracts and customer ledgers to a third party. For many owners that isn't a preference to overcome - it's the reason they don't adopt the tool at all.

What it does

Point Doculo at a folder of PDFs. Ask questions in plain language. Get answers grounded in your own files, each citing the document and page it came from.

No internet, no subscription, no data leaving the machine. It runs on CPU alone with a peak footprint of ~1.1 GB.

How we built it

Everything runs through llama.cpp as subprocesses - llama-embedding for retrieval, llama-server for generation. No PyTorch, no transformers, nothing that compiles at install time. The runtime dependencies are NumPy and pypdf. On a budget laptop, a 2 GB dependency download is its own barrier.

The pipeline: extract text per page, chunk on paragraph boundaries keeping page numbers, embed each chunk with a 36 MB quantized bge-small-en-v1.5, and store vectors in a flat NumPy matrix. Not FAISS — one business's paperwork is thousands of chunks, not millions, and a brute-force dot product over 10,000 chunks is microseconds, far below the cost of a single generated token.

We chose the model by measurement, not reputation: six models across three quantization levels, accuracy via lm-evaluation-harness at n=300, throughput and memory through the official profiler. Qwen2.5-1.5B-Instruct-Q4_K_S won - 0.723 on arc_easy, 30.73 tokens/sec, 1074 MB peak RSS.

The scoring formula shaped every decision:

$$\text{score} = 0.5\,a + 0.3\min!\left(\tfrac{\text{TPS}}{15},1\right) + 0.2\max!\left(0,\tfrac{7-\text{RAM}}{7}\right)$$

Because throughput caps at 15 t/s, extra speed earns nothing - so the ranking is mostly an accuracy ranking, provided you keep a safety margin above the cap. Q4_K_S gave us 35% more throughput than Q4_K_M at statistically indistinguishable accuracy. That margin is the design.

Challenges we ran into

Our fast laptop nearly disqualified us. The profiler compares your reported numbers against the audit's and fails a submission differing by more than 50%. Our M1 reports 68.35 t/s; the audit runs 4 vCPU on Linux from an image that compiles llama.cpp with AVX, AVX2, FMA and F16C all disabled. Submitting the native figure would have risked a fail verdict for a submission that had done nothing wrong. We now profile inside the profiler's own container capped at 4 CPU and 8 GB, which reports 30.73 t/s. This risk is invisible from the documentation - we only found it by reading the profiler's source.

A citation that named the wrong document. We uploaded two near-identical staff records disagreeing on one fact. The model answered correctly from passage 2 and said so inline — "according to Context [2]" - rather than in the USED: line our parser expected. Finding no USED: line, the code cited the top-ranked passage: the other document. A citation naming a document the answer didn't come from is worse than no citation, because it looks verified. Attribution is now evidence-based, weighting each shared term by 1/(passages containing it), so between two copies of one record the differing country decides it and the shared boilerplate barely registers.

A model that followed one instruction and dropped the rest. Llama-3.2-1B scored 0/10 on our QA set - not from weak comprehension, but because it obeyed the citation rule and nothing else, replying with the literal string USED: 1, 3 and no answer. A retry that drops the citation rule took the same model to 8/10.

Re-indexing everything on every edit. We fingerprinted file bytes, which is the obvious choice and the wrong one: re-exporting a PDF rewrites its embedded creation timestamp, changing every byte while the text is identical. Editing one clause re-embedded the whole corpus. Fingerprinting extracted text fixed it.

What we learned

Reading the evaluator's source was worth more than any optimization. It told us throughput caps at 15 t/s, that peak RSS is measured on the model alone, and that our own hardware was a liability rather than an advantage.

Small models fail in ways benchmarks don't measure. arc_easy told us which model could reason; only a grounded-QA set caught a model that answered its citation instruction and forgot the question. We built one of ten cases - eight answerable, two deliberately absent to test refusal — and it caught every real regression.

Not every good idea survives measurement. We built a relative-score cutoff to drop distractor passages, well-motivated by a case that answers at k≤2 and refuses at k≥3. Swept across the full set it only ever traded one failure for another. We deleted it and left a note so it doesn't get reinvented.

What's next

Conflict detection 0 when two documents disagree, Doculo answers from one and doesn't announce it. OCR, so scanned pages stop being invisible. And French retrieval, which matters for Central and West African SMEs.