Groundline

• 

  UI

Inspiration

A vendor security questionnaire arrives as a 400-row spreadsheet. Someone spends two weeks answering it from policy PDFs they half-remember. Vanta and Conveyor are funded companies solving exactly this problem.

Point an LLM at it and you get 30 confident answers. Some are false. In a compliance document, a confidently fabricated answer is a liability.

What it does

Groundline answered 3 of 30 questions. It refused 27. That is the honest number.

Each answer that survives carries a citation to the exact policy clause and page that supports it. Everything else is named as a gap rather than guessed at.

How we built it

Each questionnaire question becomes a property in a single JSON Schema — the question text is the field description. That schema runs against every policy PDF in the corpus through Nutrient DWS /extraction/extract.

Nutrient does the retrieval. No embeddings, no vector database, no similarity threshold to tune.

The grounding label on each citation decides the status: id_match and id_match_multiblock mean a clause was found and grounded; not_found means no policy supports it.

Two independent checks. Grounding proves the text exists in the document. It does not prove the text answers the question. Most pipelines only check the first.

Question 27 asks whether penetration tests are performed by an independent third party. A clause was found and perfectly grounded: "Conduct regular vulnerability assessments and penetration testing to identify weaknesses in the network." Real text, real citation, real page number.

It never establishes independent third party. Groundline marks it needs_review. A naive pipeline marks it answered and ships a false claim to a customer's security team.

The LLM never writes an answer. It only judges whether an already-retrieved clause is responsive. Every fact is decided deterministically.

Challenges we ran into

Confidence turned out to be the wrong signal. On a born-digital PDF, parse-stage confidence returned exactly 1.0 on every cell — trivially true, because text is read directly with no OCR. It cannot discriminate. Nutrient's own docs state confidence is relative and uncalibrated and should not be rendered as a probability. We switched to grounding labels. That negative result shaped the entire design.

Scanned documents lose ~80% of extractable content — measured against a rasterised copy of the same invoice, not assumed. Digital PDFs became a hard constraint enforced in code.

Calibrating the relevance judge. The first version rejected "access only information needed for your legitimate duties" as not being least privilege. It is least privilege, worded differently. Teaching it to judge meaning rather than vocabulary was the fix.

Accomplishments that we're proud of

The refusals are correct. q27's pentest clause is grounded and still doesn't answer the question — the system catches that, and a spot-check by hand confirmed it.

Every decision is replayable. Extractions cached by file hash, decisions logged to JSONL with actor, action and timestamp.

What we learned

Grounded is not the same as responsive. Retrieval quality and answer quality are separate axes, and conflating them is how compliance tooling produces confident false claims.

What's next for Groundline

A review UI over the DWS Viewer with bbox highlighting, so a human sees the exact clause. A signing step so the completed questionnaire ships tamper-evident. A larger corpus, and the full 400-question SIG.

Limitations

Three public policy documents, eight pages. 30 questions, not 400. Command line only — no web UI in this submission.

Built With

• json-schema
• nutrient-dws
• openrouter
• python