Inspiration
Every day, people paste things into AI tools that they should never hand to a third party: source code with API keys, customer records, patient data, contracts. Companies want their teams to use LLMs, but have no idea what data leaves the building. We wanted to make the safe path the default one.
Veil is a local privacy firewall that sits between you and any LLM provider. The provider never sees your secrets or personal data, and never receives any encryption key.
Video demo: LINK_YOUTUBE
What it does
Veil runs on 127.0.0.1 as an OpenAI-compatible proxy. Any existing app can use it by changing one line (base_url).
- Mask: a scanner detects secrets (OpenAI, Anthropic, AWS, GitHub, Google, Slack keys, JWTs, private keys,
password=pairs) and personal data (names, emails, phones, IBANs, SSNs) and replaces them with tokens like[EMAIL_1]or[OPENAI_KEY_1]. - Send: only the masked text goes to the provider, with
store: falsefor zero data retention. - Restore: tokens in the answer are replaced with the real values, locally.
- Encrypt: the token-to-value mapping is encrypted with AES-256-GCM (fresh random 96-bit nonce, authenticated context).
- Vault: conversation history is stored encrypted on disk. Its key is split with Shamir's Secret Sharing, so any 3 of 5 key holders can open it and fewer than 3 learn nothing.
- Fail closed: if masking fails, the request is blocked, never forwarded in clear.
The web UI (localhost only) has a playground, a leak report, an audit log of what the provider actually received, a vault reader, and a prompt injection demo showing that the provider has no key to reveal.
The mathematics
The vault key $K$ (256 bits) is split byte by byte over the finite field $\mathrm{GF}(2^8)$ with the irreducible polynomial $x^8 + x^4 + x^3 + x + 1$. For each secret byte $s$ we pick a random polynomial of degree $t-1$:
$$f(x) = s + a_1 x + a_2 x^2 + \dots + a_{t-1} x^{t-1}, \qquad a_i \xleftarrow{\$} \mathrm{GF}(2^8)$$
and give holder $i$ the point $(x_i, f(x_i))$ for $x_i \in {1,\dots,n}$. With $t = 3$ and $n = 5$, any three shares recover the secret by Lagrange interpolation at $x = 0$:
$$s = f(0) = \sum_{i=1}^{t} y_i \prod_{j \ne i} \frac{x_j}{x_i + x_j}$$
(in characteristic 2, subtraction equals addition). The scheme is information-theoretically secure: any $t-1$ shares are statistically independent of $s$, because for every candidate secret there is exactly one consistent polynomial. Field inverses are computed as $a^{-1} = a^{254}$ (Fermat), and multiplication is written without secret-dependent branches or table lookups, to avoid timing side channels.
How we built it
- Rust core (
seclib): AES-256-GCM, Shamir over $\mathrm{GF}(2^8)$ written from scratch, andzeroizeto wipe key material. It is exposed to Python through PyO3 and maturin, so the cryptographic execution is low level while the pipeline stays easy to extend. - Python: FastAPI proxy, scanner, encrypted vault, and the web interface.
- Interface: a clean black, white and grey web UI served on loopback, with a leak report and a vault reader.
- Tests: 22 automated security checks (nonce uniqueness, wrong key or context rejected, tampering and reordering detected, sub-threshold shares fail, duplicate shares rejected, no secret in the text sent to the provider, regression tests for false positives).
Mitigated vulnerabilities
| Risk | Mitigation |
|---|---|
| Sensitive data reaches the provider | Local masking before sending |
| Master key in prompts or logs | Key never leaves the device |
| Single point of compromise for history | Shamir 3-of-5 vault |
| AES-GCM nonce reuse | Fresh random nonce per encryption |
| Tampering or reordering of history | GCM tag plus entry index in the AAD |
| Key lingering in memory | zeroize in Rust, wiped after use |
| Provider retains conversations | store: false |
| Remote access to the UI | Localhost host check, attempt limit on the vault |
Challenges
- LLMs cannot read ciphertext, so encrypting the prompt itself is useless. We solved it with reversible tokenization: the provider gets placeholders, and the encrypted mapping stays local.
- False positives: our first password rule masked variable names like
db_password, which broke the code sent to the model. We fixed it and added regression tests. - Terminal and tooling surprises: PyO3 bindings, Rich markup swallowing
[TOKENS]in the first interface, and GitHub rejecting a workflow file.
What we learned
Real security is mostly about the boundaries: what crosses them, what is stored, and what happens when something fails. We also learned to be honest about limits instead of overselling.
Limitations (honest)
- Detection is heuristic (regex). Unusual formats, lowercase names or letter-only passwords can be missed. Production use would add entropy checks and a local NER model.
- Context stays visible to the provider, and message length is not hidden.
- While the proxy runs, the vault key sits in process memory. The 3-of-5 threshold protects history at rest, not against an attacker controlling the live process.
- Python
bytesobjects are immutable and cannot be fully zeroized. - We do not use FHE: Veil masks and encrypts the mapping and history, not the whole prompt.
What's next
A local NER model for names and addresses, entropy-based secret detection, streaming responses, and a browser extension that routes chat sites through Veil.
Made by Major Matt Razvan.
Log in or sign up for Devpost to join the conversation.