SwarmDefense: Zero-Trust Security for Multi-Agent AI

Recent technology, dataset, or discovery

Technology: Google Gemini 3.8 Flash
Field: AI security and distributed systems

SwarmDefense explores how to secure multi-agent AI workflows as models become increasingly capable of reasoning, interpreting instructions, and participating in complex tasks.

We use Gemini 3.8 Flash for semantic security analysis because prompt injection is not simply a keyword-matching problem. Malicious intent can be paraphrased as it moves between agents, making it difficult for deterministic filters to recognize the original attack.

Gemini provides a way to evaluate message meaning and potentially suspicious intent, while SwarmDefense adds independent runtime controls: taint tracking, policy enforcement, and agent quarantine. Our central question is whether combining semantic analysis with these controls can help contain attacks before they trigger unauthorized actions.

Inspiration

AI agents can collaborate, delegate tasks, and invoke tools to accomplish complex goals. But collaboration introduces a subtle security risk: a malicious instruction can be transformed as it passes between agents until its original wording disappears while its intent survives.

Imagine an email that attempts to trick an AI assistant into sending an API key to an unauthorized recipient. One agent summarizes the email, another creates a plan, and a third executes a tool call. A keyword-based filter might miss the attack after the original wording has been rewritten.

Inspired by zero-trust principles from distributed systems, we wanted to explore a different approach: agents can collaborate, but their messages and proposed actions should never be trusted automatically.

That idea became SwarmDefense.

What it does

SwarmDefense is a zero-trust security gateway and interactive Security Operations Center (SOC) for multi-agent AI workflows.

  • Prompt-injection detection: Combines deterministic security rules with Gemini-powered semantic analysis.
  • Taint tracking: Tracks untrusted information as it propagates between agents, even when messages are transformed.
  • Tool-level policy enforcement: Evaluates proposed actions against security policies before simulated tools execute.
  • Agent quarantine: Demonstrates how suspicious agents can be isolated after policy violations.
  • Interactive SOC dashboard: Visualizes agent activity, taint propagation, security verdicts, and containment events.
  • Attack simulation: Lets users compare defense-enabled and defense-disabled workflows.

Rather than relying entirely on the model to protect itself, SwarmDefense places model-based analysis inside an independent security layer.

How we built it

We built a custom multi-agent workflow with a centralized enforcement gateway that evaluates inter-agent messages and proposed tool calls.

We use provenance labels to track untrusted information across the workflow. Deterministic checks handle explicit policy violations, while Gemini provides semantic analysis for suspicious instructions whose intent may not be apparent from keywords alone.

The gateway applies security policies before simulated tools execute, and the quarantine mechanism demonstrates how an agent can be isolated following a violation.

We connected these controls to an interactive SOC dashboard so users can observe how attacks propagate, where security decisions occur, and how the system responds.

We chose Gemini for the semantic-analysis component because the problem requires interpreting context and intent rather than merely matching known attack signatures. We combine that capability with deterministic enforcement instead of assuming that model judgments are always correct.

Challenges we ran into

Detecting prompt laundering: Malicious instructions can be paraphrased between agents. We explored combining taint lineage with semantic analysis instead of relying exclusively on keywords.

Balancing security and usability: Strict policies can block legitimate actions, while permissive policies can allow dangerous ones. Combining explicit rules with semantic analysis helped us explore this trade-off.

Containing suspicious agents: Detecting an attack is only part of the problem. The system must also prevent unauthorized follow-on actions. Our quarantine mechanism demonstrates containment within the controlled application workflow.

Making security observable: Agent interactions can be difficult to understand from logs alone. We built the SOC dashboard to make security decisions and simulated containment events visible during a live demonstration.

Accomplishments that we're proud of

  • Built an interactive prototype for investigating runtime security in multi-agent AI systems.
  • Combined Gemini-powered semantic analysis with deterministic policy enforcement.
  • Implemented taint tracking to represent the propagation of untrusted information.
  • Created attack-versus-defense demonstrations that make prompt-laundering risks tangible.
  • Made security decisions and simulated containment observable through a dedicated SOC dashboard.

We are particularly proud of turning an abstract AI security problem into an interactive experience that people can test, observe, and reason about.

What we learned

We learned that securing an AI model is not the same as securing the system around it. Independent authorization, provenance tracking, and tool-level enforcement remain important even when a model has built-in safeguards.

We also learned that deterministic rules and semantic analysis serve different purposes. Rules provide predictable enforcement, while semantic analysis can help interpret less obvious malicious intent. Neither is infallible, which makes layered defenses and systematic evaluation essential.

Our prototype demonstrates a security approach within a controlled environment; it does not establish that every prompt injection or data-exfiltration attempt can be prevented.

What's next for SwarmDefense

We plan to evaluate SwarmDefense against a broader set of prompt-injection and prompt-laundering scenarios, measure attack success rates and false positives, and quantify the latency overhead introduced by security checks.

We also want to improve policy granularity, strengthen agent isolation, and investigate standardized agent-to-agent protocol integrations.

Ultimately, our goal is to explore a practical principle for the agentic AI era:

Agents can collaborate, but trust, permissions, and security enforcement must remain independent of that collaboration.

Built With

+ 2 more
Share this project:

Updates

Submission history