AgentZero

Inspiration

AI agents are becoming capable of taking real actions—not just generating text. They can read files, call APIs, send messages, upload data, modify systems, and interact with external services.

That creates a security problem: an agent can make a perfectly valid decision according to its own reasoning while still performing an unsafe action.

We wanted to build a security layer that sits between the agent and the real world. Instead of trusting the agent to decide whether its own action is safe, AGENTZERO independently evaluates the action before it executes.

Our goal was simple:

Let AI agents be autonomous without letting them become uncontrollable.

What it does

AGENTZERO is a runtime security layer for autonomous AI agents.

The agent proposes a tool call, and AGENTZERO evaluates it before execution. It analyzes factors including:

• User intent and whether the action actually matches the requested goal
• Sensitive information such as PII, financial data, credentials, and identity data
• Data provenance and source trust
• Destination trust and external data transfers
• Prompt-injection signals
• Privilege requirements
• Action impact
• Trust-boundary crossings
• Uncertainty and missing runtime information
• Security policies

It then produces a deterministic decision:

ALLOW → REVIEW → BLOCK

Every event also includes a detailed explanation showing the exact risk score for each category and why the final decision was made.

For actions requiring human approval, AGENTZERO pauses execution and waits for an explicit approval or denial.

How we built it

We built AGENTZERO as a browser-first security command center with a lightweight Python backend.

The core architecture is:

User → AI Agent → Proposed Tool Call → AGENTZERO → Policy Engine → Execution

The system separates analysis from enforcement:

AI Agent: proposes what it wants to do.

Semantic Analyzer: interprets intent, context, and suspicious signals.

Runtime Layer: provides concrete facts about files, destinations, permissions, provenance, and execution context.

Risk Engine: calculates the risk score from multiple security categories and their interactions.

Policy Engine: converts those facts and scores into ALLOW, REVIEW, or BLOCK.

The frontend was built with vanilla HTML, CSS, and JavaScript, while the backend uses Python and SQLite for local event persistence.

We also added support for different AI providers and independent models for the agent and AGENTZERO security analyzer.

Challenges we ran into

One of the biggest challenges was avoiding false positives.

For example, an action such as:

“Check my PDFs and send them to PDF.com”

should not automatically be considered an intent deviation just because it involves uploading a file to an external service.

That led us to tighten the architecture so that intent deviation is independently verified from the actual user request and proposed action, rather than blindly trusting an AI-generated score.

Another challenge was separating security signals from security decisions.

A file containing PII is a fact. An external destination is a fact. A high-impact action is a characteristic. None of those automatically means an action is malicious.

AGENTZERO therefore treats these as evidence that contributes to a broader risk calculation rather than turning every signal into a verdict.

We also had to handle human approval flows, provider-specific model discovery, uploaded documents and images, persistent event history, and safe fallback behavior when an analyzer is unavailable.

Accomplishments that we're proud of

We built more than a static security dashboard—we built a working runtime security architecture.

We're especially proud of:

• A complete ALLOW / REVIEW / BLOCK enforcement pipeline
• Independent security evaluation before tool execution
• Human-in-the-loop approval for risky actions
• Detailed per-category risk scoring for every event
• Explainable security decisions instead of opaque “AI says unsafe” messages
• Prompt-injection and data-exfiltration detection
• Runtime-aware analysis of files, destinations, provenance, and trust boundaries
• Separate models for the AI agent and AGENTZERO analyzer
• Live provider model discovery
• Persistent security event and audit history
• An attack lab for safely demonstrating adversarial scenarios
• A polished security command center designed around actual runtime decisions

Most importantly, we built the system around a principle we think is fundamental:

The agent can propose an action. It should not be the final authority over whether that action is allowed to execute.

What we learned

We learned that securing AI agents is not just about detecting malicious prompts.

The real problem is the gap between:

what the agent intends to do, what the user actually requested, and what the system is actually about to execute.

We also learned that explainability matters just as much as detection. A security system that simply says “BLOCKED” is difficult to trust and difficult to debug.

The most useful security decision is one that can answer:

What happened? What evidence did you observe? What was the risk of each factor? Which policy matched? And why was the action allowed, reviewed, or blocked?

That became one of the central design principles of AGENTZERO.

What's next for Agent Zero

Our next step is to move AGENTZERO from a strong prototype toward a production-grade runtime security platform.

We want to add:

• Deeper integration with real agent frameworks and tool registries
• Stronger runtime verification instead of relying on caller-provided metadata
• Real DLP and data-classification engines
• Destination reputation and trust intelligence
• Fine-grained identity and permission enforcement
• Persistent approval policies and organization-wide controls
• Stronger sandboxing and isolation for tool execution
• More advanced behavioral anomaly detection
• Multi-agent security monitoring
• Enterprise audit, compliance, and observability features

The long-term vision is for AGENTZERO to become the security control plane for autonomous AI—a layer that allows agents to remain powerful and useful while keeping consequential actions observable, explainable, and enforceable.

Built With

• python