GhostWire — Enterprise AI Security Middleware

• 

  Scanner-page

• 

  Dashboard-page

• 

  Incidents-page

Inspiration

Enterprise adoption of Generative AI has created an unprecedented shadow-IT vulnerability. Employees routinely paste API keys, customer PII (Protected Health Information), and proprietary source code into public LLMs.

Existing security infrastructure—DLP systems and CASB platforms—were designed for static file transfers and email. They possess zero contextual understanding of conversational LLM prompts. Post-breach detection is useless; once sensitive data hits an external model's endpoint, immediate GDPR, HIPAA, and SOC2 violations are triggered. We built GhostWire because security must move from "post-leak alerting" to "pre-execution interception."

What it does

GhostWire is an AI-native security middleware SDK. It intercepts user prompts before they leave the enterprise network, classifies the payload, and executes policy decisions in real-time.

Instead of relying on rigid regex rules, GhostWire uses a secondary LLM to police the primary LLM. It maps prompt contents to 7 distinct risk categories and automatically translates violations into regulatory frameworks.

We define the risk severity \(S\) of a prompt \(p\) using a confidence-weighted severity model. For a set of detected violation categories \(C\):

$$ S(p) = \max_{c \in C} \left( \alpha \cdot \text{Conf}(c) + \beta \cdot \text{Sev}(c) \right) $$

Where \(\text{Conf}(c)\) is the AI-generated confidence interval, \(\text{Sev}(c)\) is the predefined enterprise severity weight, and \(\alpha, \beta\) are scaling coefficients.

Based on \(S(p)\), the engine routes the prompt through our policy matrix:

How we built it

GhostWire is architected for enterprise-grade throughput and strict state segregation:

1. Classification Middleware (FastAPI): A Python backend deployed on Railway acts as the central router. It receives the payload, fires the inference request, evaluates the JSON response against the policy matrix, and dispatches the decision.
2. AI Risk Engine (Groq / Llama 3.3 70B): To achieve the required processing speed without sacrificing reasoning, we run llama-3.3-70b-versatile on Groq's LPU infrastructure.
3. SOC Dashboard (Next.js 15): A React/TypeScript application deployed on Vercel provides the Security Operations Center (SOC) interface, featuring real-time incident ledgers and severity distributions.
4. Audit Ledger (Supabase): PostgreSQL handles the immutable incident ledger with strict Row Level Security (RLS).

Challenges we ran into

• The Latency vs. Reasoning Tradeoff: Network middleware is strictly bound by time. Let \(T_{inference}\) be the latency added by the firewall. If \(T_{inference} \geq 3\text{s}\), engineers will actively bypass the system. We migrated from local quantization to Groq's LPUs, allowing us to parse complex context while maintaining \(T_{inference} \leq 200\text{ms}\).
• JSON Schema Instability under Prompt Injection: Adversarial payloads containing literal JSON strings frequently fractured the classifier's output schema. We hardened the system prompt and engineered a rigid fallback block catching json.JSONDecodeError to assign a default "WARN - Manual Review" state, ensuring the middleware never fatally crashes open.
• Asynchronous Database Bottlenecks: Initially, the API waited for the Supabase INSERT operation to resolve before returning the decision to the user, effectively doubling latency. We decoupled this: the AI classification is returned to the client instantly, while the audit logging executes in a background task thread.

Accomplishments that we're proud of

• Sub-second Forensic Analysis: We successfully built an AI-to-police-AI middleware system that operates at network-layer speeds.
• Automated Regulatory Translation: We engineered a system that doesn't just output "Data Leak"—it generates audit-ready forensic logs mapped directly to regulatory frameworks (EU AI Act, HIPAA, GDPR).
• Zero-State Architecture: The Next.js and FastAPI layers are entirely stateless, allowing infinite horizontal scaling to handle enterprise-level prompt volume.

What we learned

Building security tooling for generative AI requires abandoning static heuristics. We learned that prompting a model to act as a rigorous, unforgiving security gate requires highly constrained system instructions and strict temperature control (\(T \to 0\)). Furthermore, we validated that in security, latency is not just a UX metric; it is the primary determinant of enterprise adoption.

What's next for GhostWire — Enterprise AI Security Middleware

Transitioning from a prototype to a deployable VPC SDK. The immediate roadmap includes an air-gapped deployment architecture. The classification model (via vLLM) and the audit database will be deployed entirely within the customer's Virtual Private Cloud, ensuring zero data egress. Secondly, we are building a custom policy engine allowing CISOs to define proprietary data types and dynamically adjust the \(\alpha\) and \(\beta\) thresholds in the risk severity algorithm.

Built With

• fastapi
• groq
• llama-3
• next.js
• python
• railway
• supabase
• tailwind
• typescript
• vecel