-
-
Question asked to the system
-
Homepage
-
Response
-
User interface builders
-
Back end builders
Inspiration
For the past year and a half since returning to Kenya, I have been helping build HakiAI, a research and development company. Our flagship product is Haki Legal (https://haki.africa/). Haki Legal is a state of the art legal assistant system, designed around retrieval from embedded knowledge graph by a swarm of agents. We have hundreds of users and have processed tens of thousands of prompts. Haki has been invited to several conferences. Haki Legal looks like a successful product, destined for steady growth.
We however face a challenge in the utility of our services. State of the art agents carrying out search over a complex knowledge graph is expensive, requiring us to charge a higher premium for our services. We have thus been spending the last 6 months thinking more carefully about knowledge representation (https://haki.africa/blog/compression/) . We have been experimenting with various ways of carrying out efficient search over knowledge graphs. We have internally developed a system known as shared geometry, discussed in the next section.
This results in an efficient search scheme which allows us to build a local, cheaper version of Haki Legal, called parachichi (swahili for Avocado). The experimental demonstration shows high precision, low hallucination and the ability to quote source documents when supplied.
What we have learned
For us to be able to deliver the project, we have had to think deeply about knowledge representation. We have been researching knowledge graphs over the last year while developing schemes of automating knowledge graph construction (https://haki.africa/blog/knowledge-graphs/). We have also been carrying out knowledge compression experiments, studying knowledge injection and retrieval, a means of making knowledge retrieval and continual learning in large language models. Retrieval by shared neural geometry is our first version of a successful rollout. Inspired by BakLLava and other late fusion research, we developed a neural network geometry translator. Once the knowledge graph is encoded in our base models’ embedding geometry, the neural network simply computes the translations between the model and the knowledge graph and computes a score:
$$ \boxed{S_{\mathrm{SG}}(q,f) = 10\tanh\left(\frac{\operatorname{BM25}(q,f)}{10}\right) +\frac{2}{3} \sum_{k=1}^{3}\tanh\left( g_{\theta_k}\left(\Phi(q,f,c,n,r)\right)\right)} $$
The shared projection is tied between questions and facts:
$$\hat{\mathbf q} = \frac{W\mathbf q}{\lVert W\mathbf q\rVert_2} $$
$$\qquad \hat{\mathbf f} = \frac{W\mathbf f}{\lVert W\mathbf f\rVert_2}.$$
Here, (c), (n), and (r) represent the case, neighbourhood, and relation; g theta sub k is one of the three frozen geometry scorers. The learned correction is bounded, preventing it from overwhelming the lexical evidence.
This allows a simple scalable retrieval system facilitated by high precision search.
Building Process
We built Parachichi through an iterative research programme combining knowledge-graph construction, representation learning, and retrieval experiments. Kenyan court decisions were converted into a source-bound knowledge graph of 182 cases and 11,264 canonical facts. Python and PyTorch were used to train the shared-geometry retriever, which aligns Qwen question embeddings with relational representations. Three independently trained geometry checkpoints were frozen and ensembled.
After training, we used Causal tests like deleting target facts and inserting wrong-case distractors to confirm that answers depend on the attached graph rather than memorised language-model knowledge. An independent Python verifier was then added to reject unsupported or incomplete claims.
For natural language responses, Qwen2.5-0.5B-Instruct was adapted with LoRA using verified case facts. The adapter targeted the model’s query and value projections, was merged into the base model. We then converted this to GGUF, and quantised it for CPU inference with llama.cpp. A post-generation grounding gate checks exact fact coverage, failed drafts fall back to a deterministic verified answer, while missing evidence causes clarification or abstention.
We then packaged the application as a dependency-light Python HTTP service with a local HTML, CSS, and JavaScript interface. Our tests were carried out under Ubuntu 22.04 through WSL using CPU-only PyTorch. The resulting Parachichi system runs offline with shared-geometry retrieval, verification, and local generation in roughly 1.24 GiB of peak application memory.
Challenges faced
Building a locally competitive legal large language model retrieval involved several engineering and research complexities. Earlier experiments with knowledge injection failed stability tests. We then reworked our research around efficient retrieval. The specific finetuning recipe which worked was arrived with through several retrials. We were limited in the depth and speed of our experiments as we had access to a single NVIDIA RTX 2060 6GB VRAM.
Accomplishments that we're proud of
- The system is stable, precise and runs with low RAM requirements, which we find surprising.
- Research we carried out over the past year and a half has given us productive intuitions which helped us carry out quick experiments and rapid prototyping.
- We have developed inhouse process knowledge for “scaling down” which in our case proved more complex than scaling up.
What's next for Parachichi
We intend to scale our knowledge graph to millions of nodes and relationships, better reflecting our inhouse 250,000 legal document corpus. We are currently in negotiations with cloud companies to access the compute for this work. We estimate that this would require peak storage of 43 GB at FP32, which can be quantised further. The frontend of the system will also require modification.
We plan to use our current internal user conversations through an opt in agreement, to distil and post-train capabilities for agentic search over this knowledge graph, to develop an industrial scale downloadable legal assistant at around 4-5 GB of RAM. A doctor testing the system has developed interest in having a similar system for small scale local hospitals, as a pilot investment in healthcare.
