Inspiration
AI agents are becoming able to do more than just chat. They can now access databases, APIs, payment systems, files, and other tools to take actions for users. The problem is that an AI agent can be tricked with a malicious instruction hidden in an email, document, or webpage could make the agent perform an action the user never asked for. We wanted to solve this from a different angle, instead of trusting the AI to always make the right decision, limit what it is actually allowed to do. That is the idea behind ZeroTrust Agentic Enclave (ZT-Agent).
What it does
ZT-Agent puts a security layer between an AI agent and the systems it can access. The AI can request an action, but the action is checked against fixed security rules before it is allowed to happen. For example, an AI agent may be allowed to make payments up to $5,000. If a prompt injection tricks it into requesting a $50,000 transfer, ZT-Agent would reject the request.
How we built it
We haven't built the working prototype yet because the project is currently at the design stage. Our proposed architecture uses eBPF for kernel-level enforcement, Confidential Computing to protect the security runtime, and Open Policy Agent (OPA) for checking actions against fixed rules. Sensitive credentials would also be kept away from the AI agent. We plan to start with a simple working prototype where an AI agent attempts both allowed and unauthorized actions, then test whether ZT-Agent can reliably block the unauthorized ones.
Challenges we ran into
The main challenge was figuring out where the security boundary should actually be. We didn't want to build another system that simply asks another AI whether an action is safe. We wanted the final decision to be based on rules that cannot be changed by the agent itself. We also had to research what technologies such as eBPF, Confidential Computing, and remote attestation can realistically do and where they fit into the system.
Accomplishments that we're proud of
We turned the idea of "securing AI agents" into a specific architecture. Instead of trying to make the AI impossible to trick, we designed the system around the assumption that the AI can be tricked, but the attacker should still be unable to make it perform actions outside its allowed permissions.
What we learned
We learned that AI security is not only about protecting the AI's prompts or responses. A major part of the problem is what the AI is allowed to do after it makes a decision. We also learned that different security technologies solve different parts of the problem, so they need to be carefully combined rather than added just because they sound useful.
What's next for ZeroTrust Agentic Enclave (ZT-Agent)
We learned that AI security is not only about protecting the AI's prompts or responses. A major part of the problem is what the AI is allowed to do after it makes a decision. We also learned that different security technologies solve different parts of the problem, so they need to be carefully combined rather than added just because they sound useful.
Log in or sign up for Devpost to join the conversation.