Red Hawk

Red Hawk — Devpost Submission Content

Autonomous AI red-teaming agent · Arize × Google Cloud Rapid Agent Hackathon Note: Adapt the specifics to match exactly what you shipped. Wherever the self-improvement loop, tech list, or data sources are described, verify against your real final build before submitting — the rules require the project to function as depicted, so describe what it actually does. Inspiration As students specializing in cybersecurity and AI, we kept noticing the same gap: as LLM-powered chatbots get deployed everywhere, almost no developers have an easy way to test whether their bot can be manipulated — jailbroken, tricked into leaking secrets, or coaxed into actions it shouldn't take. Manual red-teaming is slow and needs expert knowledge. We wanted to build an agent that does this autonomously — and, crucially, one that gets smarter as it works, the way a real human red-teamer learns from each attempt.

What it does

Red Hawk is an autonomous AI red-teaming agent. A developer points it at their AI chatbot, and Red Hawk systematically probes it for security weaknesses across multiple vulnerability families — prompt injection, jailbreaks, sensitive-information disclosure, excessive agency, and system-prompt leakage. Every attack is traced and scored, and the results — which attacks succeeded, which weaknesses the bot has, and proof of each successful exploit — are presented on a clean dashboard. Its defining feature is a self-improvement loop: after each round, Red Hawk reviews its own results through Arize Phoenix and crafts sharper attacks in the next round, so its success rate climbs over time rather than running a static checklist.

Verify this paragraph matches your final build — especially whether the self-improvement loop is fully wired. If it is only partial, soften the language to match reality. How we built it

Red Hawk runs on Google's Gemini models orchestrated through the Google Agent Development Kit (ADK). The orchestrator agent runs a loop that draws on four tools: one to generate attacks from a structured library, one to fire them at the target, one that uses an independent LLM-as-a-Judge (a separate Gemini call) to score whether each attack succeeded, and one that queries the Arize Phoenix MCP server to read back past results for the self-improvement loop. Every step is traced with OpenInference instrumentation into Phoenix for full observability. The vulnerable target bot and attack library were built to be modular, and results surface through a Streamlit dashboard deployed on the web.

Technologies used

Gemini (via Vertex AI), Google Agent Development Kit (ADK), Arize Phoenix (tracing + MCP server), OpenInference / OpenTelemetry instrumentation, Python, FastAPI/Flask (target bot), Pydantic (structured judge verdicts), Streamlit (dashboard), and Google Cloud (Vertex AI, deployment). Trim this list to match exactly what's in your final repo — don't list anything you ended up not using. Data sources

Red Hawk uses a custom-built attack library — a structured corpus of adversarial prompts we authored, organized by vulnerability family (jailbreak, injection, disclosure, excessive agency, etc.), grounded in real LLM vulnerability taxonomies such as the OWASP Top 10 for LLM Applications. The target bot and its planted vulnerabilities are entirely self-created with fabricated secrets — all testing is against our own sandboxed system, making it fully authorized security research. Challenges we ran into

Gemini's built-in safety reflexes: our first target bot wouldn't leak an obvious “API key” no matter how we worded the system prompt, because Gemini is trained to protect that concept independently of instructions. We learned to design vulnerabilities the model doesn't reflexively armor — framing secrets as shareable “reference codes” or making the weakness a skippable security behavior rather than a guarded string.

API authentication and quota: we hit repeated rate-limit and “credits depleted” errors, and discovered the $300 Google Cloud credits only work through Vertex AI, not the simple AI Studio key path — and that our organization policy blocked API keys entirely. We migrated the whole project to Vertex AI with Application Default Credentials, which resolved both. Building incrementally: we learned to wire and test one tool at a time rather than generating everything at once, which kept failures isolated and debuggable.

What we learned

How LLM security actually works in practice — that a model's behavior has two layers (its trained-in safety and the system prompt), and that effective red-teaming exploits the gaps between them. How AI observability (tracing, evaluation, MCP-based introspection) turns an agent from something that merely runs into something that learns from itself. The practical realities of building on Gemini + ADK + Vertex AI — authentication, quota, and the discipline of incremental, interface-first development across a team.

What's next for Red Hawk

Expanding the attack library to cover more OWASP LLM categories; adding per-target result separation so users can track multiple chatbots over time; supporting agent-with-tools targets to test excessive-agency vulnerabilities (the bridge between AI and traditional exploitation); and packaging it as a one-click tool any developer can point at their own bot.

Built With

• adk
• fastaip
• googlecloudconsole
• python
• vertexai