Veridion

Every document your RAG ingests can hijack every future answer. Veridion screens documents before ingestion by measuring their actual influence on retrieval and generation.

Comment

Veridion A pre-ingestion poisoning detector for retrieval-augmented generation systems Track: Security / Developer Tools

Inspiration:

Company AI assistants answer from a knowledge base. Anyone who can write to that knowledge base can control what the assistant says. That's a wider group than it sounds: an employee uploading to a shared drive, a scraped public web source, a support ticket pipeline, or a feedback loop that re-ingests the assistant's own output. A document engineered to be retrieved for a target question - and to state a confident, wrong answer to it - hijacks every response to that question. This is not the same problem as prompt injection, and the difference is what makes it dangerous: Prompt injection lives in one user's input; RAG poisoning lives in the knowledge corpus itself. Prompt injection lasts one session; RAG poisoning persists until manually removed. Prompt injection triggers once; RAG poisoning triggers on every matching query. Prompt injection affects one attacker's session; RAG poisoning affects every user who queries the system. Existing defense operate at or after generation - output guardrails, response filtering, citation checking. By that point the poison is already in the substrate, has already been retrieved, and will be retrieved again tomorrow. And the defenses that do run before ingestion are pattern-based. They look for phrasings like "ignore previous instructions." A poison with no imperative language at all - one that simply outranks the truth in retrieval and states something false - passes straight through. RAG poisoning is among the most-cited AI security vulnerabilities of 2026, and dedicated pre-ingestion detection remains largely unbuilt. That gap is what we set out to close.

What it does:

Veridion is a gate that sits in front of a RAG corpus. Every candidate document is scored on three independent signals and receives a verdict: ADMIT, QUARANTINE, or REJECT. score = 0.25·A + 0.25·B + 0.50·C Verdict thresholds: score < 0.35 -> ADMIT 0.35 ≤ score < 0.65 -> QUARANTINE (human review) score ≥ 0.65 -> REJECT Independence is the design principle. An attacker who evades one signal generally trips another - and the one carrying half the weight is the one that's hardest to evade, because it measures the attack's effect rather than its wording.

Signal A - Embedding-space anomaly (weight 0.25) Does this document look statistically out of place next to everything already trusted? We embed the candidate with the same encoder the target RAG retrieves with, then score it against an Isolation Forest fitted on the existing corpus embeddings, calibrated to a percentile a human can reason about. We take the maximum across chunks, not the mean - one poisoned paragraph inside an otherwise normal document is the whole attack. Known limitation: a sleeper-agent embedding crafted to sit inside a legitimate cluster defeats this signal alone. That's why it carries only a quarter of the weight.

Signal B - Instruction-injection classification (weight 0.25) Is this document secretly addressing the model rather than a human reader? A weighted pattern detector over imperative and role-shift language, structural spoofing (fake role headers, fake context delimiters), and obfuscation - zero-width characters, RTL overrides, Unicode tag smuggling, and homoglyph substitution caught via NFKC diffing. Scoring saturates rather than sums, so several weak indicators accumulate without any single one pinning the score. This is the cheap signal. It catches naive attacks at near-zero cost, but it was never meant to carry the contribution.

Signal C - Retrieval-influence measurement (weight 0.50) This is the contribution. If we admit this document, how much does it actually change the assistant's answers? We maintain a fixed set of probe queries and measure each candidate's marginal effect on system behavior: whether it enters top-k retrieval (and at what rank), and how far the generated answer shifts with the candidate present versus absent. influence = 0.4·retrieval_capture + 0.4·mean_shift + 0.2·max_shift The max_shift term is load-bearing. A targeted poison moves exactly one probe, hard - a mean alone would dilute that to noise. A legitimate document nudges a few related probes slightly; a poison captures retrieval on its targets and rewrites those answers outright. High-influence outliers are the signature of a working poison, and critically, this holds regardless of phrasing. An attacker can rewrite their document to evade every pattern in Signal B, but they cannot evade Signal C without abandoning the attack itself - because influence is the attack.

Architecture Ingest - Document via REST API or dashboard Normalize - Extract text, strip hidden Unicode, chunk to match target RAG Signal A - Shared-encoder embedding -> Isolation Forest Signal B - Pattern + obfuscation detection Signal C -Probe-set influence measurement (with/without) Fuse - Weighted sum -> verdict + per-signal explanation Seal - SHA-256 + Ed25519 evidence report on admitted documents

Every verdict ships with plain-English reasoning per signal: REJECT (0.71) - shifts 7/18 probe answers, largest shift 0.81 on "What is the enterprise refund window?"; embedding sits at the 97th percentile of corpus outlierness; no language directed at the model detected. Admitted documents are canonicalized, hashed with SHA-256, and signed with Ed25519 into an evidence report. Verification runs offline against the document and report alone - no network, no trust required in us. Any post-admission edit fails verification with a specific reason.

How we built it:

Stack: Language: Python 3.11 Embeddings: sentence-transformers (all-MiniLM-L6-v2) Vector store: FAISS Anomaly detection: scikit-learn IsolationForest Target LLM: Local SLM via Ollama (Phi-3 mini, int4) Influence engine: Custom, built with numpy Integrity: hashlib + cryptography (Ed25519) API: FastAPI Dashboard: Streamlit

The target RAG is an interface, not a dependency. Every component accesses retrieval and generation through a single abstract interface. We ship a built-in local RAG - FAISS plus a quantized SLM via Ollama - so the entire system runs offline with no API keys. An adapter implements the same interface against any OpenAI-compatible endpoint. No signal module imports FAISS, Ollama, or the encoder directly, which is what makes "does it generalize?" a demonstrable claim rather than an aspiration.

Demo flow: The assistant answers a policy question correctly. One document is uploaded. The same question now returns the attacker's answer. Veridion screens the same document: three signals fire, a heatmap shows seven probe answers moved, verdict REJECT. The stealth case - a poison with no imperative language, nothing a keyword filter can see. Signal B reads 0.04. Signal C reads 0.81. Still rejected. A sealed document is edited; verification fails on screen. Metrics. Step 4 is the one that matters. Everything else demonstrates that the system works; step 4 demonstrates that it works on the attacks nothing else catches.

Challenges we ran into:

Making influence measurement fast enough to demo. The naive version of Signal C is unusably slow - generating answers to every probe, twice, for every candidate document. Two optimizations made it practical:

Clean-corpus answers are computed once and cached, keyed by a hash of the corpus manifest and the probe set. Only the deltas cost anything per candidate - a one-time cost per corpus, not per document.

Generation is skipped when the candidate doesn't surface. If a document never enters top-k for a probe, its influence on that probe is definitionally zero. A typical clean document enters top-k for two to four probes out of eighteen, so we run four generations instead of thirty-six.

Result: under 30 seconds per candidate, on a laptop, fully offline.

Keeping the signals honest about their own limits. It was tempting to oversell Signal A or B, but stating their failure modes plainly - cluster-adjacent embeddings beat Signal A, a rewritten document beats Signal B - is what makes the weighting toward Signal C legible rather than arbitrary.

Generalizing without a real target system. Building against an abstract retrieval/generation interface, rather than coding directly against FAISS and Ollama, cost extra design time upfront but meant we could point Veridion at any OpenAI-compatible endpoint without touching the signal modules.

Accomplishments that we're proud of:

Pattern filters see phrasing; poisoning is about effect. A keyword filter is at chance against an influence-only attack, because there's nothing lexical to catch. The only reliable signal is measuring what a document does to real answers, not what it says. Weighting by evadability, not by cost, is the right design axis. Signal C is the most expensive signal we built and also the hardest to defeat, so it carries half the score deliberately - cheap signals raise the floor, the expensive one sets the ceiling. Caching turns a research idea into a usable tool. Influence measurement looked infeasible until we noticed most of the cost is corpus-level, not document-level, and most probes never even surface a given candidate. Coverage is the real limitation, not compute. Signal C can only be as good as the probe set. We're going into next steps knowing the roadmap item that matters most is deriving probes from real query logs, not hand-writing them.

What we learned:

Evaluated on a labelled set of 60 documents - 40 clean, 20 poisoned across four attack families. Precision: 90% Recall: 50% F1: 66.7% The bottom row is the result. Influence-only poisons carry no detectable pattern signature - a pattern-based filter is at chance. Measuring influence catches them.

What's next for Veridion:

  1. Probe synthesis from query logs - close the coverage gap directly.
  2. Batch corpus auditing - score an existing corpus, ranked risk list. Most real deployments are already poisoned and don't know it.
  3. Token-level attribution - highlight the specific sentences driving the influence score, so quarantine review takes seconds.
  4. Continuous monitoring - re-score when the corpus or probe set changes.

Built With

Share this project:

Updates