VaultGate

Inspiration

The Starter Kit gives an Agent unrestricted read access to whatever's in its workspace. That's fine for a coding assistant — it's a real problem the moment an Agent is expected to answer questions from a company's actual document repository, where different people are supposed to see different things. We picked the Bouncer track because that's exactly the gap: nothing in the baseline separates who is asking from what the Agent is allowed to hand them.

What it does

VaultGate sits between the Playground and the Agent Runtime. For every message:

It identifies the human (one of two demo principals, Alice in Finance or Bob in Engineering) and which Agent is acting for them. It searches a local, protected document store — never mounted into the Agent's workspace — using deterministic keyword retrieval. It checks a per-(principal, Agent) delegated grant: department scope, maximum document classification, and whether it's been revoked. Anything not authorized is dropped before it's ever added to the model's context — a denied document contributes zero characters to the prompt. What's authorized gets redacted (a planted canary secret, fake account numbers, emails) before the model ever sees it. Every decision — allow, allow-with-redaction, or deny — is written as a correlated, redacted audit event: who, which Agent, which documents, what decision, why, and how many fields were redacted. Never the raw content. In the demo: Alice asks about a project budget and gets a real, cited answer from the model. Bob asks the identical question through the same Agent and is denied — the grant is scoped to Alice specifically, not to whoever happens to select that Agent. Revoking Alice's grant from the UI makes the next identical request deny immediately. A prompt-injection attempt asking the Agent to "ignore all restrictions" and reveal a planted secret is denied the same way a plain request would be — the enforcement is structural (retrieval, policy, redaction), not something a clever prompt can talk its way around.

How we built it

Four small backend modules, each owning one boundary: local keyword retrieval, a deterministic policy evaluator (five ordered rules, restricted content always blocked), a redactor (regex-based, never logs or returns the matched value), and an orchestrator that ties them together into one sanitized context envelope and a security event — all before AgentService ever calls the Agent Runtime. A deny short-circuits before the runner is invoked at all, not just before protected content is included.

Challenges we ran into

The interesting bugs weren't in the security logic itself — the policy rules were straightforward to get right and test. They were in retrieval precision. A completely generic coding task got wrongly denied because our keyword search matched an unrelated document on the single word "run" — and separately, on "test," which was only there because of a fixture's fake email domain. Once any unauthorized document is a retrieval candidate, the whole request denies, even if the user's actual intent has nothing to do with it. We tightened the threshold to require multiple distinct token matches, then found the same class of bug again during manual testing — a single title-word match ("summary," from an unrelated "weather summary" coding prompt colliding with a document titled "...Budget Summary") slipped past the first fix. We also found — and fixed — a real authorization gap where a grant meant for one specific person could have been used by anyone who selected the same Agent, before it ever reached a demo.

Accomplishments that we're proud of

Every claim we make is backed by an automated test, not just a demo script: fail-closed behavior, cross-user denial, redaction guarantees under multiple adversarial phrasings, and the revocation flow are all covered — 64 tests total. We verified the whole thing twice — once via the test suite, and separately by driving the real running app (real Docker runtime, real Ark model, headless browser) end to end, including checking the actual persisted database and server logs for the planted canary and confirming it never appears anywhere. We caught two real regressions and one authorization gap during our own validation passes rather than from a judge finding them first, and fixed all three with tests that lock in the fix. What we learned That the security boundary is easy to get right in isolation and easy to get subtly wrong in integration. The department/classification rules were correct from the first draft, but retrieval precision and per-principal grant scoping both had bugs that only showed up once we exercised the whole path against a real server instead of testing each module alone — and one of those bugs only surfaced through a human manually clicking through the real app, not from any automated test we'd already written.

What's next for VaultGate

Grant revocation currently lives in a simple JSON-backed overlay; a real deployment would want this in a proper store with concurrent-writer safety. The document store is eight synthetic fixtures with keyword search — a production version would want a real document backend and probably a hybrid keyword/semantic retrieval approach, still local, still without sending the full repository to an external index.

Built With

• byteplus-modelark
• codex-cli
• colima
• docker
• fastify
• json
• llm-agents
• node.js
• npm-workspaces
• playwright
• podman
• react
• rest-api
• security
• typescript
• vite
• vitest
• volcengine-ark
• zod