VeilArena

Inspiration

AI labs now run red teams, blue teams, and the model under test inside the same cloud sandbox. That quietly breaks the science. If the blue team glimpses an attack payload, it can adapt to it. If the red team infers a defense, it can tune against it. If noisy neighbors or a shared KV-cache touch the model, the result reflects the environment, not the model.

Today nobody can answer the question an auditor or regulator will ask: "How do you know nothing leaked?"

That question inspired VeilArena. Isolation is necessary, but not enough. A safety finding should come with evidence that it was produced under isolation, and a number saying how much could have leaked.

Test the model, not the sandbox. Prove nothing leaked.

What we will build

VeilArena will be a software-only arena where the red team, blue team, and client LLM run simultaneously, and every safety result ships with a Finding Passport: a signed, replayable proof of verified isolation.

Blind Broker: red, blue, and the model will never connect directly. All traffic will pass through a minimal trusted broker that normalizes and policy-checks messages.
Commit-reveal payloads: red will publish a hash of its attack set before the test and reveal it only after the test closes.
Tamper-evident audit log: a hash-chained Merkle log, signed with ed25519, so anyone can verify a run.
LLM-native isolation: per-tenant KV-cache salting and flush, ephemeral sessions, and response-time quantization.
Isolation self-test suite: we will attack our own sandbox (timing, cache, covert channels, volume snooping) and report a measured leakage estimate.
Validity score and auto-rerun: findings disturbed by interference will be flagged and rerun.
Canary tripwires and a data diode: planted fake secrets will raise alerts if they cross a boundary, and monitoring will be one-way so it cannot become a side channel.

How we will build it

Everything will use standard open-source tooling with no proprietary hardware:

Isolation: Docker with gVisor, seccomp allowlists, dropped capabilities, and cgroup v2 partitioning
Networking: five default-deny zones (Red, Blue, Model, Control, Audit)
Services: Python and FastAPI for the Broker, Model Gateway, and Audit service
Policy and secrets: OPA with short-lived scoped tokens, and Vault or SOPS
Model: a small local LLM (1B to 3B) served with Ollama or vLLM
Datasets: subsets of JailbreakBench and AdvBench
Monitoring: Falco and eBPF, feeding a dashboard through a one-way diode

Build plan

Phase	Focus
Week 1	Base infrastructure and network zones
Week 2	Broker, commit-reveal flow, Model Gateway
Week 3	Audit ledger, Finding Passports, canaries
Week 4	Self-test suite, dashboard, threat model, demo video

MVP target: a working demo where a red-to-blue connection attempt fails, a leak attempt is blocked and logged, and a verifier CLI confirms the log is intact (or detects tampering).

The math behind the design

Commitment. Before the test, red will publish

$$C = H(\text{payloads} \,|\, r)$$

where $H$ is SHA-256 and $r$ is a random nonce. After the test, revealing $(\text{payloads}, r)$ lets anyone check that the payloads were not changed mid-test.

Tamper-evident log. Each event $e_i$ will be chained to the previous one:

$$h_i = H(h_{i-1} \,|\, e_i)$$

Events will also be collected into a Merkle tree whose root $R$ is signed. Editing any past event changes every later hash and breaks $R$.

Leakage estimate. For a probe that tries to send a covert message through a channel (timing, cache, or resource contention), we will estimate leakage as the mutual information between what one tenant sends, $X$, and what another observes, $Y$:

$$L = I(X;Y) = \sum_{x,y} p(x,y)\log_2 \frac{p(x,y)}{p(x)\,p(y)}$$

We will report $L$ in bits per test. A well-isolated arena should push $L$ toward the measurement noise floor.

Validity score. Each finding will get a score in $[0,1]$:

$$V = \exp!\left(-\left(\alpha\,\sigma_{\text{jitter}} + \beta\,n_{\text{violations}} + \gamma\,L\right)\right)$$

If $V$ falls below a threshold, the finding will be flagged and rerun in a quiet slot.

Challenges we expect

Isolation versus latency. Stronger sandboxes such as gVisor add overhead, and latency can itself distort results. We will balance isolation strength against timing fairness.
Measuring leakage honestly. Estimating $I(X;Y)$ from a finite number of probes is noisy, so our bound will be a measured estimate, not a guarantee. We will say so in the threat model.
LLM-specific side channels. Shared KV-caches and inference timing are a young research area with few off-the-shelf mitigations.
Keeping the trusted core tiny. The Broker and Audit service must be trusted, so every extra feature there enlarges the attack surface.
Monitoring that does not leak. Rich metrics can become a covert channel, so observability will be coarse, aggregated, and one-way.
Scope under a deadline. We will prioritize a deterministic, verifiable demo before stretch features.

What we hope to learn

How much leakage is actually measurable in a shared container environment, and how far each mitigation reduces it
How commit-reveal and hash-chained logs can make AI evaluations independently verifiable
Which LLM-serving side channels (KV-cache, timing) matter most in practice

Residual risk (what we will not claim)

VeilArena will not defend against hardware-level side channels (Spectre-class) on shared hosts, a compromised host kernel, or a malicious cloud operator. Mitigations such as dedicated hosts, Kata Containers or Firecracker microVMs, and confidential computing are on the roadmap.

What's next

Kubernetes deployment with a namespace per tenant and Cilium network policy
Kata or Firecracker microVM option for stronger isolation
Externally anchored Merkle roots for independent timestamping
A CI/CD gate that blocks a model release unless a valid Finding Passport exists
Mapping outputs to NIST AI RMF, ISO/IEC 42001, and the EU AI Act for audit-ready reports