Multi-Agent Anti-Phishing (MA2P) Framework

MA2P: A self-evolving, multi-agent shield for phishing—fact-checks claims, flags social engineering, and explains each verdict, adapting via adversarial self-play.

Comment 1

Multi-Agent Anti-Phishing (MA2P) Framework

Revolutionizing Email Security Through Specialized LLM Agents

The Problem & Our Disruptive Solution

The Crisis: Generative AI has weaponized phishing attacks. Cybercriminals now generate 10,000+ convincing phishing emails per hour, causing $10.5 billion in annual losses. Traditional keyword-based filters are obsolete against AI-crafted deception. User privacy is severely compromised as more people fall prey to unintentionally giving up their personal data.

Our Innovation: a Multi-Agent Anti-Phishing (MA2P) Framework designed to combat the evolving threat of phishing attacks. Leveraging the power of Large Language Models (LLMs) and an adversarial self-play training methodology, this system provides a robust, transparent, and adaptive solution for detecting and analyzing suspicious digital communications. Unlike traditional methods that rely on static signatures or opaque black-box models, MA2P employs a committee of specialized AI agents that work in concert to dissect potential threats from multiple angles, offering detailed explanations for their verdicts. This approach significantly enhances detection accuracy and provides users with actionable insights into the nature of the threat.

Features

MA2P is built with a suite of powerful features designed to provide comprehensive and transparent phishing detection:

  • Multi-Agent Architecture: At its core, Agentic-PhishNet employs a sophisticated multi-agent system. An Orchestrator Agent acts as the central coordinator, receiving incoming text (e.g., email content) and intelligently distributing it to specialized sub-agents. It then aggregates their individual analyses and confidence scores to form a unified, final verdict. This distributed approach ensures a thorough examination of all aspects of a potential phishing attempt.

  • Fact-Verification with Retrieval Augmented Generation (RAG): The Fact-Verification Agent is specifically designed to scrutinize the factual claims within the input text. This agent is now significantly enhanced with a Retrieval Augmented Generation (RAG) system. This means that beyond its internal knowledge, the agent can perform real-time web lookups to verify information, cross-reference details, and identify inconsistencies. For instance, it can validate company names, contact information, financial figures, and procedural claims against external, credible sources. This RAG capability makes the Fact-Verification Agent exceptionally robust against novel and context-specific phishing lures, as it can adapt its verification process based on the latest available information.

  • Language Analysis: The Language Analysis Agent focuses on the subtle, yet critical, linguistic and stylistic cues that often betray a phishing attempt. It analyzes elements such as manufactured urgency, threatening or manipulative tone, grammatical anomalies, generic greetings, and various social engineering tactics. By understanding the persuasive and deceptive language patterns, this agent provides a crucial layer of defense against even well-crafted scams.

  • Adversarial Self-Play Training: A key innovation of Agentic-PhishNet is its continuous learning mechanism. Our agents are not trained on static datasets but are instead refined through an adversarial self-play methodology, inspired by the SELF-REDTEAM framework Liu et al., 2025. In this paradigm, an 'Attacker' LLM continuously generates increasingly sophisticated phishing scenarios, while our 'Defender' agents (Fact-Verification and Language Analysis) learn to detect them. This iterative process ensures that the system constantly adapts and improves its detection capabilities against novel and evolving phishing techniques, staying ahead of malicious actors.

  • Transparent & Explainable AI: Agentic-PhishNet prioritizes transparency. The user interface provides detailed reasoning for its verdicts, going beyond a simple phishing/non-phishing label. It highlights suspicious phrases directly within the input text and offers agent-specific insights into why certain content was flagged. This explainability builds user trust and educates them on how to identify sophisticated phishing attempts.

  • Intuitive Web Interface: The project includes a user-friendly web interface developed with React.js. This interface allows users to easily submit text for analysis, view the real-time processing by the agents, and visualize the detailed analysis results, including highlighted sections and confidence scores. The design focuses on clarity and ease of use, making advanced phishing detection accessible.

  • Scalable & Modular Design: Agentic-PhishNet is built with a modular architecture, allowing for easy integration of new agents or enhancements to existing ones. This design ensures that the framework can adapt to future threats and expand its capabilities to cover various forms of digital communication beyond email, such as text messages, social media posts, or document analysis.

Technical Stack

Our full-stack demo application leverages a combination of robust technologies for its backend processing, LLM interactions, and user interface.

Backend (Python)

The backend serves as the computational core, handling API requests, orchestrating agent interactions, and managing LLM calls.

  • Django: A high-level Python web framework that enables rapid development of secure and maintainable websites. In Agentic-PhishNet, Django is used to build the RESTful API that interfaces with the frontend, manages incoming requests, and routes them to the appropriate agent logic.
  • openai Library: The official Python client library for the OpenAI API. This library is crucial for making requests to OpenAI's large language models, which serve as the foundation for our Fact-Verification and Language Analysis Agents.
  • numpy: A fundamental package for scientific computing with Python. It is used for numerical operations, particularly in processing and aggregating confidence scores and other quantitative data from the LLM agents.
  • requests: A popular HTTP library for Python, used for making web requests.
  • dotenv: A Python library that loads environment variables from a .env file. This is used for securely managing sensitive information such as API keys without hardcoding them into the codebase.
  • concurrent.futures: Part of Python's standard library, this module provides a high-level interface for asynchronously executing callables. Specifically, ThreadPoolExecutor is utilized in the Orchestrator Agent to run the Fact-Verification and Language Analysis Agents in parallel, significantly speeding up the analysis process.
  • ddgs: A Python library for programmatic access to DuckDuckGo search. In Agentic-PhishNet, it underpins the WebRetriever module, which performs web queries to support Retrieval-Augmented Generation (RAG). Search results are returned as titles, snippets, and URLs, then deduplicated and re-ranked by relevance before being passed to the Fact-Verification Agent as external evidence for claim validation.

Frontend (JavaScript/React)

The frontend provides the interactive user interface, allowing users to submit content for analysis and visualize the results.

  • React.js: A declarative, component-based JavaScript library for building user interfaces. React enables the creation of dynamic and responsive web applications, providing a smooth and intuitive experience for users interacting with Agentic-PhishNet.
  • Vite: A frontend tooling that provides an extremely fast development experience. Vite is used as the build tool for the React application, offering rapid hot module replacement (HMR) and optimized production builds.

LLM Models

  • gpt-4o-mini: This specific OpenAI model is configured as the default base model for both the Fact-Verification and Language Analysis Agents. Its efficiency and performance make it suitable for real-time analysis within the framework. The choice of LLM is configurable, allowing for flexibility to integrate other models as they become available or as specific performance requirements dictate. The use of a smaller model shows that our framework is potentially feasible with smaller, local models.

Complex Challenges Conquered

Advanced AI Training Obstacles:

  • Reward Stagnation: Overcame mode collapse in adversarial training where agents stopped learning—solved through penalty systems and forced exploration mechanisms
  • Multi-Agent Synchronization: Engineered coordination between competing agents while maintaining performance requirements

Engineering:

  • Latency vs. Accuracy Trade-offs: Optimized parallel processing architecture to achieve sub-30 second response times without sacrificing detection quality
  • Memory Management: Solved token usage optimization challenges to make the system cost-effective for enterprise deployment

Achievements

Research-to-Production Success:

  • Implementation: Successfully adapted theoretical SELF-REDTEAM framework into working phishing detection system
  • 80%+ Detection Accuracy: Outperformed traditional methods on sophisticated AI-generated phishing emails in comprehensive benchmarks
  • Self-Improving System: Achieved continuous learning capability—agents get better over time through adversarial evolution

Technical Innovation Milestones:

  • Dual-Agent Architecture: Created novel integration of linguistic and factual analysis with game-theoretic decision fusion
  • Explainable AI Excellence: Developed character-level highlighting system providing precise reasoning for every detection
  • Open Source Impact: Created a reusable framework that advance the field of adversarial AI applications

Key Insights & Potential Market Impact

Technical Breakthroughs:

  • Adversarial AI Mastery: Learned that successful adversarial training requires careful balance between exploration and exploitation—our solution to reward stagnation could revolutionize RL applications
  • Multi-Agent Orchestration: Discovered that game-theoretic integration dramatically outperforms simple voting mechanisms in multi-agent systems

Market Disruption Potential:

  • $10.5B Market Opportunity: Our solution directly addresses the massive annual cost of phishing attacks with superior AI-powered detection
  • Enterprise Scalability: API-first architecture enables rapid integration with existing email infrastructure (Outlook, Gmail, Exchange)
  • Competitive Advantage: Self-evolving capability means our system stays ahead of threats while competitors rely on static rule updates

Built With

+ 12 more
Share this project:

Updates