Canary

Canary is an autonomous AI agent that red-teams other AI systems for prompt injection — live, in a real browser — and only reports a finding once the target itself proves it.

Comment

Inspiration

I learned to code through vibe coding — building by iterating with AI rather than starting from textbooks — and along the way I fell in love with a specific niche: bug bounty hunting, pentesting, and LLM vulnerability research. I'm always chasing what's new. When prompt injection kept showing up as OWASP's #1 risk for LLM applications, I couldn't stop thinking about one question: what would it take to build an AI agent that hunts these bugs the way a real human pentester does — not by throwing canned payloads at a target, but by actually reasoning about it?

That's the dream behind Canary: a multi-agent system that thinks like a professional bug hunter — logical, methodical, adaptive — with one non-negotiable rule: zero false positives. A finding only counts when the target itself proves it, never when the model just says so.

What it does

Canary opens a real browser, recons an AI target, classifies how it's vulnerable (direct or indirect/ stored injection), generates attack probes across a taxonomy of real prompt-injection technique classes, adapts when a probe fails, and only reports a finding once it's independently confirmed — by a matched ground-truth flag, an accepted submission, or a reproduced result in a fresh session.

How I built it

This is version one, built in about a week for OpenAI Build Week. I used Codex with GPT-5.6 as my primary build environment — the first time I've worked with Codex, and it completely changed how I iterate: I could describe a phase of the agent's reasoning loop, get a working implementation, and verify it against a real self-test before trusting it, all in one back-and-forth. I split the work by difficulty — simpler, well-scoped tasks went to Luna, and the deep-thinking, complex engineering (the core agent loop, the technique taxonomy, the verification logic) went to Terra.

I tested it live against two authorized, no-login public labs — Wraith's Oracle of Whispers (indirect/ stored injection) and Lakera's Gandalf (an 8-level, self-verifying password game) — because I wanted proof, not a mockup.

Challenges I ran into

Getting the agent to actually reason instead of pattern-match was the hardest part — early versions would bring the wrong strategy to the wrong target, or confidently call something solved when it wasn't. I spent real time building a verification layer that never trusts the model's own opinion: ground truth always comes from the target — a real flag pattern, or the target's own acceptance message. I also hit real infrastructure problems along the way — gateway API keys that authenticated but returned empty completions, browser timing bugs, a multi-part probe that occasionally crashed the live UI — and treated every one as something to fix and prove, not paper over.

What's next

This is genuinely just the first version. I'm going to keep developing Canary — more OWASP LLM categories beyond injection, broader target coverage, and eventually a path for teams to point it at their own pre-launch chatbots and copilots before shipping them. I'm excited I got to use Codex for the first time building something in this space, and I plan to keep going.

Built With

Share this project:

Updates