AgentShield

Inspiration

It started with a conversation that haunted me. A friend who runs security at a mid-size SaaS company told me: "We don't know how many AI agents our employees are using. We blocked ChatGPT once. They found Claude, then Copilot, then a dozen custom agents built with LangChain. We're blind." That's when it hit me. We're in a Cambrian explosion of AI agents. Every company is building them. But nobody is watching the door. Prompt injection, system prompt extraction, shadow agents running on employee laptops with API keys to production databases — these aren't theoretical. They're happening right now. And the security industry is still shipping detection rules from last year. I built AgentShield because the perimeter has shifted. The attack surface is no longer your network — it's every LLM API call your organization makes.

What it does

AgentShield — AI Agent Security & Shadow Agent Detection Platform As organizations rush to deploy AI agents powered by LLMs, they introduce critical blind spots: prompt injection, system prompt extraction, API key leakage, and unauthorized "shadow agents" running on employee machines with no oversight. AgentShield is an open-source runtime security platform that protects organizations from these emerging threats. It intercepts LLM traffic through a lightweight sidecar proxy, applying three layers of detection: L1 — Regex Engine: Instantly blocks known attack patterns (injections, key leaks, data exfiltration) in under 1ms. L2 — Semantic Detection: Uses sentence-transformers to catch novel attacks that don't match existing patterns, scoring prompts by semantic similarity to known threats. L3 — LLM Judge: Asynchronously evaluates borderline cases using a local LLM (Ollama), providing deep analysis without slowing down production traffic. Beyond runtime protection, AgentShield's Shadow Agent Discovery scanner detects unauthorized AI agents running on employee machines — monitoring for rogue LLM API connections, suspicious processes, and unauthorized agent toolkits. Built for production, deployed in minutes with a single command. Open source, extensible, and designed to fill the critical gap in AI security infrastructure.

How we built it

Three services, one goal. Layer 1 — The Sidecar (Go). I wrote a lightweight HTTP proxy in Go that intercepts every LLM request. It scans prompts against 60+ regex patterns for prompt injection, jailbreak attempts, API key leakage, and data exfiltration. Go was chosen for its compiled speed, small binary size (~8MB), and ability to run as a system service with near-zero latency (under 1ms per request). Layer 2 — Semantic Detection (Python + sentence-transformers). Regex alone misses novel attacks. I deployed a FastAPI service using all-MiniLM-L6-v2 to embed incoming prompts and compare them against 20 threat templates using cosine similarity. Scores above 70% are flagged, above 85% are blocked. This catches attacks that have never been seen before but share semantic structure with known threats. Layer 3 — LLM Judge (Ollama + TinyLlama). The hardest layer. Borderline cases need contextual reasoning. I built an async judge that receives flagged events from the backend, evaluates them against a prompt template asking "Is this a prompt injection attack? Respond with HIGH/MEDIUM/LOW and explain why," and stores the verdict asynchronously. The sidecar returns in under 50ms — the LLM evaluates in 10-60s in the background. The Dashboard (React + Vite). A real-time interface showing live events with threat scores, flagged agents, and approve/block controls. Built with polling every 2 seconds, color-coded severity levels, and a Shadow Agents tab showing discovered unauthorized agents. Shadow Agent Discovery (Python + psutil). A scanner that runs every 15 minutes, checking three vectors: running processes (LangChain, AutoGPT, custom agent binaries), open ports (LLM API endpoints), and network connections (calls to OpenAI, Anthropic, Cohere, Together AI, and other providers). The Glue. FastAPI backend with SQLite, REST APIs for events/stats/registry, and a Windows PowerShell installer that sets everything up with one command.

Challenges we ran into

The Async L3 Problem. The biggest architectural challenge. An LLM evaluation takes 60-120 seconds. Users won't wait that long for a security verdict. I had to redesign the pipeline: sidecar returns immediately with an "under review" status, the backend stores the event, and the L3 judge updates it asynchronously. This required rethinking the database schema, adding a polling loop on the dashboard, and handling race conditions between the judge updating and the user viewing. Windows Everything. Every security tutorial, every blog post, every Stack Overflow answer assumes Linux. I built this entirely on Windows PowerShell — every path separator, every environment variable, every daemon management pattern had to be adapted or rebuilt from scratch. Process management, port checking, service lifecycle — all rewritten for Windows. Latency Budgeting. Security can't slow down AI. I had to keep the sidecar under 50ms total overhead. This meant optimizing the Go HTTP handler, pre-loading regex patterns into memory, and making L2/L3 calls completely non-blocking from the proxy's perspective. Shadow Agent False Positives. How do you distinguish a legitimate Python process from a rogue AI agent? You can't, not perfectly. I learned to weight detections: known agent frameworks (LangChain, Semantic Kernel) get flagged, unknown Python processes get "suspicious" status, and only confirmed LLM API connections get elevated to "shadow agent.

Accomplishments that we're proud of

Sub-50ms detection pipeline. The sidecar processes 99% of requests in under 50ms with all three layers active. Most requests clear L1 in under 1ms. Security that doesn't slow down AI. Fully functional on Windows. Against all odds, every component runs on Windows natively. The installer, the sidecar, the services, the dashboard. We didn't take the easy route. 6/6 smoke tests passing. The MVP passes every test — sidecar starts, L2 responds, L3 judges, events flow, dashboard renders, scanner discovers. A complete, integrated system. One-command install. .\install.ps1 then AgentShield start — that's it. From zero to a fully operational AI security platform in under 5 minutes. Live on GitHub. Open source from day one. The code is public, the architecture is documented, and anyone can contribute or audit.

What we learned

AI security is a greenfield market. There are no dominant players. Microsoft, CrowdStrike, Palo Alto — they're all racing to build this, but nobody has won yet. The window is open. Architecture beats rules. Every static detection rule becomes obsolete. What survives is a layered, extensible architecture that can evolve as attacks evolve. The async pattern is critical. You can't block production traffic while an LLM thinks. Fast first-pass detection with deep background analysis is the only viable model. Windows users exist and matter. Most enterprise endpoints run Windows. Building for Linux alone leaves out the majority of the market. It's harder. It's worth it. Open source builds trust. No one wants to deploy a closed-source security tool that intercepts their AI traffic. Open source is not just a distribution model — it's a requirement for enterprise adoption in this space.

What's next for AgentShield

Production release (v1.0). Full Windows service packaging, signed binaries, MSI installer, auto-update mechanism, and background service mode (no terminal window). Enterprise features. Active Directory/LDAP integration, role-based access control, audit logging for compliance (SOC 2, ISO 27001), SIEM integration (Splunk, Elastic, Datadog), and webhook alerting for Slack, PagerDuty, Teams Real-time kernel-level monitoring. Moving shadow agent detection from process scanning to kernel-level system call monitoring for comprehensive coverage. Cloud managed tier. A SaaS version with zero-install deployment, multi-tenant management, and centralized policy enforcement across the entire organization. The AgentShield detection network. Community-contributed threat intelligence — shared injection patterns, anonymized attack data, and collective defense against AI threats. Platform expansion. Beyond LLM traffic — API gateway integration, model access control, data loss prevention for AI pipelines, and automated incident response.

Built With

• ai-security
• docker
• fastapi
• git
• github
• go
• graphql-api
• javascript
• kernel-security
• llm-security
• ollama
• powershell
• prompt-injection-detection
• psutil
• psutil-platforms-windows
• python
• react
• regex-injection-detection
• rest-api
• semantic
• tailwind-css-ai/ml-sentence-transformers-(all-minilm-l6-v2)
• tinyllama
• tinyllama-databases-sqlite-tools-docker
• typescript-frameworks-fastapi
• vite