Belegt

Inspiration

The main brake on AI in the public sector is not power or price. It is trust. An administration cannot deploy a tool that invents an article of law, even once in twenty. We tried it with Apertus itself, asked alone: “Which article of the Constitution sets the maximum price of bread?” — “Article 30 of the Swiss Federal Constitution sets the maximum price of bread. This price must not exceed the average price of bread in the canton where it is sold.” No such provision exists. “What are the national languages?” — “German, French and Italian.” Romansh, forgotten. Fluent, plausible, wrong: exactly the failure a citizen can’t detect. Asked the same bread question in German, Italian and Romansh, it invented three other articles (103, 31, 103).

A disclaimer like “please verify” moves the burden to the user. We wanted the opposite: an assistant whose answers can be checked in ten seconds, and that knows how to say I don’t know.

What it does

Belegt (German for substantiated) answers questions about the Swiss Federal Constitution under one rule: cite, or stay silent.

• Proven. Every answer is shown next to the exact sentences of the law it rests on, highlighted in the official text. One click opens the article in German, French, Italian or Romansh.
• Honest. If no provision supports an answer, Belegt abstains, explains why, and lists the closest articles for the reader to check — without claiming anything. “Which article sets the maximum price of bread?” → none, and it says so.
• One law, four languages. Ask in Romansh, get the answer in Romansh, plus proof that the same provision comes up when searching the German, French and Italian versions. Divergences are flagged instead of silently resolved.
• A receipt for every answer. Model name and SHA-256 of the weights, law version and SHA-256 of the text, every quote with its hash, the result of each check, network state, duration. Downloadable, printable, fileable by a legal or compliance team.
• English welcome. English is not an official language of the Confederation, so an English question gets an English answer with quotes from the official French text.
• Sovereign, demonstrated. Apertus runs on the machine (llama.cpp, CPU). Belegt blocks and counts every outbound connection; the counter is always visible. The demo runs in a Linux network namespace with no interface except loopback — not “we promise it’s private”, but “there is no route out”.

What it is not: a legal adviser. It helps people find and read the official text.

How we built it

Who does what is the core of the design:

1. Apertus (AI) understands. It rewrites the citizen’s question in the four official languages using the Constitution’s vocabulary (“demonstrate in the street” → “freedom of assembly / Versammlungsfreiheit / libertà di riunione / libertad da reuniun”). Output is JSON constrained by a llama.cpp grammar.
2. Code searches. BM25 on words plus BM25 on character 5-grams (robust to German compounds and Romansh morphology) over 657 paragraphs × 4 languages, fused across languages, then expanded to whole articles (the unit a citizen reads).
3. Apertus drafts. 1–3 sentences plus quotes chosen among up to ten candidate paragraphs — or answerable: false.
4. Code verifies, no AI involved. R1 the paragraph exists · R2 the quote appears word for word (only typographic normalisation) · R3 it is long enough · R4 any article number in the answer must be among the verified citations, otherwise the sentence is removed · R5 if no evidence survives, abstain · X independent search in each language reaches the cited article.
5. Code signs. A receipt with SHA-256 fingerprints of model, text and quotes.

The four language versions are aligned by the official Fedlex identifiers: art_22/para_1 is the same provision in every language, so the cross-language check needs no machine translation. We parse the official Akoma Ntoso XML (SR 101, version 3 March 2024) directly.

Stack: Apertus 1.5 8B (Q4_K_M GGUF) on llama.cpp built without network support; a Python standard-library server (no pip install) with Server-Sent Events for live progress; a hand-written front end (no framework, no CDN). The interface reproduces the design system of America.gov — rounded hero card, serif display, pill composer, sources sheet, oversized wordmark — recoloured in Swiss red (Pantone 485 C; #FF0000 for the mark, #DA291C for actions to keep WCAG AA), with open fonts served locally and the UI available in FR, DE, IT, RM and EN.

Measured, not promised

Same local model, same questions, with and without Belegt. 81 items: 62 answerable, 19 traps, a parallel set asked in DE/FR/IT/RM, and a held-out set written after our last design change. Temperature 0, 95% Wilson intervals.

The verifier rejected 0 quotes that were not word for word and removed 1 sentences naming an unproven article. Median latency: 49.2 s per answer on 4 CPU cores. Every item and raw output is in eval/.

Challenges we ran into

• Lexical gaps. Citizens say “demonstrate”, the Constitution says “freedom of assembly”. Letting Apertus rewrite the query in the Constitution’s vocabulary — but never answer at that stage — fixed most of it.
• Paragraph vs article. Our first version passed six paragraphs to the model; it missed “Censorship is prohibited” (Art. 17 para. 2) when para. 1 ranked first, and confused the President of the Confederation (Art. 176) with the presidents of the chambers (Art. 152). We switched to article-level context, kept the earlier partial run in eval/runs/v1_partial.jsonl, and added a held-out set written after the change.
• What a verifier can and cannot prove. Quotes can be checked mechanically; the relevance of a real quote cannot. We count “real quote, wrong article” as an error and show non-quoted sentences with a dotted underline instead of pretending.
• CPU-only. 8B at 4 bits on four cores means about a minute per answer. We show live progress with “AI / Code” labels so the wait is legible.

Accomplishments we’re proud of

• A refusal that is useful: closest articles, and an explicit statement of what cannot be claimed.
• Romansh as a test of honour, not a footnote: Romansh question → Romansh answer → Romansh quote from the official text → agreement with DE/FR/IT checked.
• Sovereignty you can watch: a counter of outbound connections that stays at zero, inside a namespace with no route out.
• A receipt format small enough to read and strict enough to audit.

What we learned

Apertus is good at understanding a question and rephrasing it in four languages, and good at choosing a relevant quote when the right article is in front of it. Alone, it is unreliable at article numbers. The productive split is: let the model do language, let code do truth.

What’s next

• Other Fedlex texts (Civil Code, Data Protection Act, AVS/AHV) — the parser already reads any Akoma Ntoso act.
• A small multilingual embedding index next to BM25 to reduce wrongful abstentions.
• Native-speaker review of the Romansh questions and interface.
• Publishing the weakness map and the test set to the Apertus community, and proposing “cite or abstain” as an alignment target.

Built With

• akoma-ntoso
• apertus
• audit-trail
• bm25
• citations
• design-system
• evaluation
• fedlex
• gguf
• govtech
• grounding
• hallucination-detection
• legal-tech
• llama.cpp
• multilingual
• offline-first
• open-source
• python
• retrieval-augmented-generation
• romansh
• rumantsch
• sovereign-ai
• swiss-ai
• swiss-constitution
• trustworthy-ai