Provenance Firewall for AI Agents

Inspiration

While every modern AI agent platform logs what an agent did, almost none can tell you why it was allowed to do it. Standard observability is purely post-mortem. We realized that as autonomous agents ingest data from untrusted external sources, traditional perimeter security isn't enough. We needed a proactive system that understands the causal chain of an agent's actions and stops execution before a malicious prompt or file can trigger data exfiltration.

What it does

Provenance Firewall is a provenance-aware execution firewall for AI agents. Rather than just logging actions, it tracks the context entering a model call and tags it by source (operator, agent-derived, or untrusted-external).

1. Taint Propagation: This "taint" automatically propagates to any subsequent action the agent attempts.
2. Action Enforcement: Privileged actions—like network egress or out-of-workspace file writes—are actively denied or escalated for human approval at the container boundary if their causal chain contains tainted sources.
3. Unified Observability: The exact same provenance graph powering the firewall generates our audit traces, meaning active containment and rich observability fall naturally out of a single model.

How we built it

We bootstrapped the project using the CodeJam starter, giving us a solid foundation of Fastify, React, and Codex-in-containers.

1. Engine & Schema: We defined our core provenance and policy schemas centrally in types.ts, powering a deterministic taint checker and policy engine.
2. Firewall & Mocking: We built a default-deny egress firewall that intercepts agent requests, tested against a real local mock server.
3. UI: We built a correlated trace store paired with a "jump-to-failing-span" UI, allowing operators to instantly visually trace exactly where a policy violation originated.
4. Testing: We engineered a custom chaos harness that injects failures into the agent environment and programmatically asserts complete containment.

Challenges we ran into

Propagating taint deterministically across asynchronous model calls and container boundaries was highly complex. AI agents naturally synthesize data, making it difficult to definitively trace an untrusted input through a black-box LLM generation and map it accurately to an outgoing network request without triggering false positives.

Accomplishments that we're proud of

Getting the containment and observability to fall out of the exact same provenance graph was a major win. We didn't have to build a separate logging tool and security tool; the causal chain drives both. We are also incredibly proud of the chaos harness, which reliably proved our default-deny architecture holds up against active injection attacks.

What we learned

We learned that attempting to secure LLMs purely by filtering their output is a losing battle. True agentic security has to happen at the execution boundary, and context/provenance is vastly more reliable than pattern-matching for stopping malicious payloads.

What's next for Provenance Firewall for AI Agents

We plan to expand our source-tagging to integrate directly with popular frameworks like LangChain and AutoGen. We also want to implement finer-grained, per-variable policy schemas and move the default-deny egress firewall out of the mock environment into a production-ready Kubernetes operator.

Built With

• agents
• docker
• modelark
• orchestration
• react
• typescript