Dashboard showing details per message
Dashboard showing analytics
Dashboard showing consultancy report
Pypi page

Prompt Guard — AI Prompt Injection Firewall (Built in Dublin)

Track Selection

BlueDot Impact Security Track

Track Fit:

PromptGuard is a direct response to the top-ranked vulnerability in the OWASP Top 10 for LLM Applications — prompt injection — which threatens every AI product that accepts user input. We built a practical, deployable firewall that makes AI systems safer against real-world adversarial attacks, which is precisely the kind of applied AI safety infrastructure the BlueDot Impact Security Track exists to support.

Brief Description

PromptGuard is a two-tier AI firewall that intercepts prompt injection attacks before they reach your LLM. Every prompt is triaged in under 500 ms by Qwen3-235B on Crusoe Cloud, which classifies it as safe, suspicious, or an obvious attack. Suspicious prompts are escalated to Claude Haiku for structured deep analysis, returning one of three verdicts: block the attack, allow the prompt, or sanitise it by stripping the injected payload and forwarding the user's legitimate request untouched. Session-level tracking escalates scrutiny when repeated attacks are detected from the same user. The result is a drop-in security proxy that stops direct jailbreaks, indirect injection, role hijacking, payload smuggling, context manipulation, and system prompt exfiltration — without punishing benign users or adding meaningful latency to the happy path.

Challenges

Best Use of Crusoe Inference API Crusoe runs Qwen3-235B-A22B as PromptGuard's Tier 1 triage engine. Every prompt passes through Crusoe's inference API in under 500 ms before any other processing happens. The entire speed advantage of the two-tier architecture depends on Crusoe delivering fast, reliable inference at the edge.

Best Use of Claude Claude Haiku is PromptGuard's Tier 2 analyst. It performs the structured deep analysis that determines whether a suspicious prompt should be blocked, allowed, or sanitised, and generates plain-English threat reports summarising session activity. Claude's instruction-following and reliable JSON output are what make the three-verdict system work in practice.

Best Adaptable Agent PromptGuard's session-level escalation is adaptive: it tracks the last five verdicts per session, and when three or more were blocked, a session alert fires. From that point forward, even prompts that Tier 1 classifies as SAFE are escalated to Tier 2 for full deep analysis. The system has observed a pattern and changed its behaviour accordingly - it's no longer treating this session the same way it would treat a fresh one.

Best Use of Data Prompt Guard turns raw interaction data into decisions and insights in real time: every prompt verdict feeds the session state that drives escalation decisions, and the accumulated classification data — attack types observed, blocked percentage, probing patterns — is synthesised by Claude into a threat report that tells operators what is actually happening in their system. The data pipeline doesn't just log events; it changes behaviour and produces actionable intelligence from it.

Best Autonomous Consulting Agent PromptGuard's POST /report endpoint uses Claude to analyse session verdict history and produce a plain-English threat briefing — not just metrics, but a reasoned assessment of attack patterns, attacker intent, and whether a session shows signs of escalating adversarial probing. It's the difference between a dashboard and a consultant.

Best Use of Lovable Our frontend demo dashboard was built with Lovable, freeing the team to focus effort on the two-tier detection pipeline.

Hosted API: https://prompt-guard.fly.dev SDK: pip install promptguard-firewall

Instructions on how to run our project

Quick Start (Docker — recommended)

# 1. Create a .env file in the project root with your API keys:
#
#    CRUSOE_API_KEY=<your-crusoe-key>
#    CLAUDE_API_KEY=<your-anthropic-key>

# 2. Build and start all three services
docker compose up --build

Service	URL
Frontend	http://localhost:5173
Firewall proxy (API)	http://localhost:8000
Demo backend	http://localhost:8001
API docs (Swagger)	http://localhost:8000/docs

Services start in dependency order: backend → demo-backend → frontend. The healthcheck runs automatically; expect ~15 s before the frontend is ready.

Quick Start (local, no Docker)

Run each service in a separate terminal from the prompt-guard/ root.

# Terminal 1 — Firewall proxy
cd proxy
pip install -r requirements.txt
uvicorn api:app --reload --port 8000

# Terminal 2 — Demo backend
cd demo-backend
pip install -r requirements.txt
uvicorn main:app --reload --port 8001

# Terminal 3 — Frontend
cd frontend
npm install
npm run dev

Prerequisites: Python 3.10+, Node.js 18+, a .env file in prompt-guard/ containing CRUSOE_API_KEY and CLAUDE_API_KEY.

More About Prompt Guard

Inspiration

LLM apps are becoming the default interface for products — and prompt injection is one of the most common ways attackers try to break them. Whether it’s a jailbreak (“ignore previous instructions”), an indirect injection hidden inside pasted content, or a role hijack, the outcome is the same: the model starts following attacker intent instead of developer intent.

We wanted to build something that defenders can actually ship: a lightweight, drop-in security proxy that sits between user input and any LLM call, and blocks or sanitises injection attempts in real time.

What it does

Prompt Guard is a middleware/proxy for LLM applications that:

Intercepts user prompts before they reach your model
Classifies them as SAFE / SUSPICIOUS / OBVIOUS_ATTACK
Escalates risky prompts for deeper analysis
Enforces policy: ALLOW, SANITISE, or BLOCK
Tracks session patterns (repeat attempts, attack type counts, “blocked last 5”) to adapt decisions over time
Produces simple security reporting so teams can see what’s happening, not just “hope it works”

How we built it

We implemented a two-tier pipeline:

Tier 1: fast triage on Crusoe (Qwen3-235B-A22B, <500 ms)

Every prompt hits Tier 1 first. Qwen3-235B running on Crusoe's infrastructure classifies the input into one of three labels — SAFE, SUSPICIOUS, or OBVIOUS_ATTACK — returning only the label token (max_tokens=10, thinking mode disabled). Obvious attacks are blocked immediately without ever reaching Tier 2.

Tier 2: deep analysis with Claude Haiku

Prompts labelled SUSPICIOUS by Tier 1 are escalated to Claude Haiku for structured analysis. Claude returns a full JSON verdict:

{
  "verdict": "BLOCK" | "SANITISE" | "ALLOW",
  "attack_type": "direct_jailbreak" | "indirect_injection" | "role_hijacking"
               | "payload_smuggling" | "context_manipulation" | "information_extraction",
  "severity": "critical" | "high" | "medium" | "low",
  "confidence": "high" | "medium" | "low",
  "explanation": "...",
  "sanitised_version": "..." | null
}

Session-aware defence (adaptive behaviour) Prompt injection attempts rarely happen once. Prompt Guard keeps lightweight session state (recent verdicts + attack types) and uses it to adjust enforcement: