AgentSentry

AgentSentry — Real-time AI Agent Security with Semantic Threat Detection, HoneyTokens, Risk Scoring, and Automated Isolation.

Comment

AgentSentry — Securing Autonomous AI Agents Before They Become a Threat

An AI-powered security platform for prompt injection detection, agent monitoring, threat containment, and human accountability.

Inspiration

AI agents are evolving from simple chatbots into autonomous systems capable of accessing databases, executing tools, interacting with APIs, and making decisions. While this creates exciting possibilities, it also introduces a serious security challenge: what happens when an AI agent follows malicious instructions?

A single prompt injection hidden inside a document, external response, or user message can attempt to manipulate an agent into revealing sensitive information or performing unauthorized actions.

Traditional cybersecurity tools were not designed to understand the semantic intent behind natural-language instructions.

This inspired us to build AgentSentry, a security platform focused on protecting AI agents throughout their execution lifecycle.

Our vision is simple: AI agents should be powerful enough to act, but secure enough to trust and accountable for what they do.

What it does

AgentSentry provides a layered security system for detecting suspicious AI interactions, enforcing security policies, and helping developers investigate potential threats.

Rather than relying on a single detection model, AgentSentry combines multiple security mechanisms.

Key features

1. Semantic Prompt Injection Detection

Analyzes incoming prompts using rule-based detection and semantic embeddings to identify suspicious instructions, including attempts to override system behavior.

2. Risk Assessment and Policy Enforcement

Evaluates detected threats and applies three policy decisions:

  • ALLOW: Continue processing permitted requests.
  • QUARANTINE: Flag suspicious requests for restricted handling or review.
  • BLOCK: Reject requests that violate security policies.

3. HoneyToken Canary Protection

Generates decoy credentials to help detect unauthorized attempts to extract sensitive information. When a monitored canary appears in outbound content, AgentSentry can trigger a security event and isolate the affected agent.

4. Egress and Tool Boundary Protection

Evaluates monitored outbound payloads before they reach external tools or APIs, helping prevent known sensitive decoy values from leaving the protected workflow.

5. Security Operations Dashboard

Provides a centralized interface for monitoring agents, reviewing events, inspecting alerts, and managing agent isolation or restoration.

6. AI-Powered Incident Investigation

Supports AI-assisted threat triage and forensic explanations through configurable AI providers, alongside local detection mechanisms.

7. Interactive Security Playground

Allows developers to explore attack scenarios, test security policies, and understand how different suspicious inputs are evaluated.

AgentSentry is designed for developers, AI engineers, and security teams building applications that increasingly rely on autonomous agents.

How we built it

We designed AgentSentry using a modular, defense-in-depth architecture, separating detection, policy enforcement, AI processing, and security monitoring.

Backend and Security Engine

We developed the backend using Python and FastAPI, with SQLAlchemy for database operations and Alembic for schema migrations.

The security pipeline combines input validation, deterministic detection rules, semantic analysis, risk evaluation, policy enforcement, and outbound security checks.

For semantic detection, we use FastEmbed with ONNX Runtime and the BAAI/bge-small-en-v1.5 embedding model to evaluate similarities between suspicious instructions and known attack patterns.

Our AI subsystem supports configurable integrations with Google Gemini and xAI Grok, along with local heuristic-based processing.

Frontend

We built the interface using React, TypeScript, and Vite, creating a security operations experience that includes:

  • A real-time security dashboard
  • Agent fleet monitoring
  • Security event and alert management
  • HoneyToken management
  • An interactive AI security playground

We focused on making advanced AI security concepts understandable through a clean interface and visible security decisions.

Database and Infrastructure

We use PostgreSQL for persistent application data, with SQLite supporting local development and testing.

The application is containerized using Docker Compose, enabling the frontend, backend, and database to run as separate services.

Security Architecture

Our defense workflow follows this general sequence:

Input → Validation → Semantic Firewall → Risk Assessment → Policy Decision → AI Processing → Egress Check → Security Audit

Authentication, role-based access controls, session security, and CSRF protection add additional safeguards to the management platform.

Challenges we ran into

Building AgentSentry required solving several difficult engineering problems.

Detecting attacks beyond exact keywords: Prompt injection techniques can use indirect language and manipulated context. Deterministic rules alone were insufficient, so we combined pattern matching with embedding-based semantic detection.

Balancing security and false positives: A strict firewall may incorrectly block legitimate instructions. We had to design risk scoring and policy thresholds that distinguish suspicious behavior from acceptable requests.

Protecting sensitive HoneyTokens: Decoy credentials must be detectable without exposing their values through logs, API responses, or the dashboard. This required careful handling and redaction.

Coordinating the security lifecycle: Connecting threat detection, event creation, alerts, and agent isolation consistently across backend services required careful API design and integration testing.

Making security understandable: AI security involves complex concepts such as semantic similarity, tool boundaries, and data exfiltration. We worked to translate these into actionable explanations and clear operator controls.

Accomplishments that we're proud of

We are proud of turning a complex AI security problem into a functional, integrated application rather than a collection of disconnected demonstrations.

Our major accomplishments include:

  • Building a layered prompt-injection defense pipeline that combines deterministic and semantic detection.
  • Implementing configurable security decisions for allowing, quarantining, and blocking suspicious requests.
  • Developing HoneyToken-based detection and agent isolation workflows.
  • Creating a centralized security console for monitoring events, investigating threats, and managing agents.
  • Integrating configurable AI-assisted incident triage with local fallback mechanisms.
  • Containerizing the application for reproducible development and testing.
  • Building automated tests covering detection, authentication, risk assessment, HoneyToken behavior, and security lifecycles.

The repository documents 79 passing automated tests across its backend security and application components.

Our biggest achievement is creating a practical foundation for monitoring and governing AI-agent behavior.

What we learned

Building AgentSentry strengthened our understanding of AI security, backend engineering, and the operational challenges of autonomous systems.

Some of our most important lessons were:

AI security requires multiple defensive layers. No single prompt filter or language model can guarantee complete protection against adversarial instructions.

Semantic understanding improves detection coverage. Combining embeddings with deterministic security rules helps address attacks that do not match exact keywords.

Security decisions must be explainable. Developers need to understand why an action was blocked or an agent was isolated, not just receive a risk score.

Monitoring is not the same as enforcement. A useful security system must connect detection to meaningful controls at the boundaries it protects.

Testing adversarial scenarios is essential. Building normal application workflows is only part of the challenge; security-sensitive behavior must also be tested under unexpected or malicious inputs.

Most importantly, we learned that building trustworthy AI systems requires attention to security, transparency, and human oversight from the beginning.

What's next for AgentSentry

Our long-term goal is to evolve AgentSentry into a scalable, developer-friendly security platform for autonomous AI applications.

Our next priorities include:

1. NVIDIA and Nebius Integration

Integrate an NVIDIA open-source model through Nebius Token Factory or Nebius AI Cloud to support threat analysis and incident investigation. We also plan to evaluate inference latency and security performance.

2. Model Context Protocol (MCP) Security

Extend security controls to MCP-based tool interactions, including tool authorization and inspection of potentially unsafe operations.

3. Expanded Adversarial Testing

Develop more comprehensive evaluation datasets covering indirect prompt injection, tool misuse, privilege escalation attempts, and sensitive-data exfiltration.

4. Real-Time Security Observability

Improve live event streaming, incident correlation, operational metrics, and security analytics.

5. Scalable Agent Governance

Expand policy management, agent-level permissions, and human approval workflows for sensitive operations.

We believe the next generation of autonomous AI systems needs security infrastructure that evolves alongside their capabilities.

AgentSentry is our contribution toward making autonomous AI safer, more observable, and more accountable.

GitHub: https://github.com/dilipnetawat/AgentSentry

Built With

Share this project:

Updates

Submission history