Spectra

An AI agent for next-gen ethical hacking and browser automation, providing a secure, scalable, and interactive platform for both cybersecurity professionals and accessible web browsing.

Comment

Tagline

Your AI-powered co-pilot for browsing, hacking, and accessibility.

About the project

Inspiration

The world of digital automation is fragmented. Powerful web automation frameworks, sophisticated security scanners, and data transformation tools exist in separate silos. We were inspired to break down these walls. We saw a significant problem: traditional automated security tools are blind to the UI-driven, interactive context where many modern vulnerabilities hide, while existing AI agent frameworks lacked the robust, sandboxed environments needed to safely interact with the real world. Our vision was to create a unified, cloud-native platform where a team of AI agents could collaborate, equipped with a diverse toolset to tackle complex, multi-step tasks that mimic human-level interaction.

What it does

Spectra is an advanced, cloud-native platform that brings a hierarchical team of AI agents to life, allowing them to perform complex browser-based tasks in a secure and scalable manner. It operates in two unique modes, demonstrating its innovative dual-purpose design:

  1. Hacking Mode: This mode provides a powerful co-pilot for security professionals. A user can set a high-level pentesting objective (e.g., "Log into this application and test for XSS in the user profile page"), and Spectra's agents will:

    • Plan: Autonomously devise a multi-step strategy.
    • Interact: Navigate login forms, handle dynamic UI elements, and explore the application.
    • Test: Use an integrated suite of Kali Linux tools and CyberChef recipes to discover and verify vulnerabilities.

  2. Normal Mode: This mode transforms Spectra into an AI-powered accessibility tool. Users can issue natural language commands (text or voice), and the AI agent will execute browsing tasks on their behalf, significantly lowering digital barriers for users with disabilities.

The entire system is deployed on Google Kubernetes Engine (GKE), where a central router dynamically provisions a complete, isolated session environment (browser, agents, and tools) for each user. This ensures security, scalability, and efficient resource management.

How we built it

Spectra is a testament to strong technical execution, combining a modular microservices architecture with a sophisticated, AI-driven core. Our code is organized into clean, independent services, each well-documented within its own directory.

Advanced Use of Google's Agent Development Kit (ADK)

We leveraged the core concepts of the ADK to build a truly collaborative, multi-agent system, moving beyond single-agent implementations.

  • Hierarchical Agent Collaboration: Our architecture features a Planner Agent that decomposes high-level goals into tactical steps. It then delegates these steps to specialized executor agents: a Clicker Agent for UI manipulation, a Pentest Agent for security scanning, and a CyberChef Agent for data transformation. This showcases a clear and effective use of multiple AI agents working together to achieve a complex task.
  • Reasoning and Planning: We utilized ADK's PlanReActPlanner to give our Planner agent the ability to reason about its environment (via screenshots), form a coherent plan, and adapt as the task progresses.
  • Custom Tool Integration via MCP: We built custom tools and exposed them to the agents using the Model Context Protocol (MCP). Our pentools service runs a Flask API fronting a Kali Linux container, with its functions (like nmap_scan or sqlmap_scan) exposed via a fastmcp server. This demonstrates a clean, decoupled approach to extending agent capabilities, a core principle of the ADK.

Technical Implementation & Cloud-Native Architecture

Our solution is built for scalability and security, deployed on Google Kubernetes Engine (GKE).

  • Microservices: The project is broken down into containerized microservices (Agent, Browser, Pentools, CyberChef, RTMP Server, Router, Redis), each with its own Dockerfile and clear responsibilities. This modularity ensures the code is clean, maintainable, and independently deployable.
  • Dynamic Session Provisioning: A central Node.js Router acts as the control plane. Using the @kubernetes/client-node library, it authenticates users via Firebase and dynamically creates a dedicated Kubernetes namespace and a multi-container "Session Pod" for each user. This is a highly efficient and scalable approach to providing isolated environments.
  • State Management & Routing: The router uses Redis to manage session state and http-proxy-middleware to intelligently route traffic to the correct user-specific services within the cluster.
  • CI/CD: Our GitHub Actions workflow automatically builds and pushes all service images to Docker Hub, ensuring a streamlined and efficient development lifecycle.
  • Architecture Diagram: The project README.md includes detailed, interactive architecture diagrams created with Mermaid, clearly defining the data flow for both our Docker Compose and Kubernetes deployments.

Challenges we ran into

  • Vision Model Precision for UI Automation: An early challenge was the coordinate precision of vision models like Gemini. A request to "click the login button" would often yield coordinates that were slightly off. We overcame this by engineering more descriptive prompts and building a more resilient interaction logic that could handle minor inaccuracies.
  • Dynamic Per-User Service Routing on Kubernetes: Routing a user's request to a service inside a dynamically created namespace was a complex networking challenge. We engineered our Node.js router to act as an intelligent proxy that queries Redis for the user's active namespace before forwarding traffic, creating a seamless experience.
  • Graceful Session Teardown and Asynchronous Tasks: Simply deleting a Kubernetes namespace could lose data. We had to design a graceful shutdown process where session recordings could finish processing and uploading to GCS before the underlying resources were terminated. This required a carefully orchestrated cleanup flow.
  • The Learning Curve of Google ADK: As a powerful and relatively new framework, the Agent Development Kit presented a steep learning curve. We invested significant time in understanding its patterns for hierarchical agent design and integrating custom toolsets via MCP, which ultimately became one of the project's greatest strengths.

Accomplishments that we're proud of

  • Building a True Multi-Tenant AI Platform on GKE: We are incredibly proud of designing and deploying a system where multiple users can run completely isolated, resource-managed AI agent sessions simultaneously. The dynamic provisioning and cleanup demonstrate a high level of technical execution.
  • Implementing a Functional Multi-Agent Hierarchy: We successfully created a system where a Planner agent reasons, strategizes, and delegates tasks to specialized agents. This showcases a deep and practical application of the ADK's multi-agent capabilities.
  • The Innovative Dual-Mode Solution: Creating a single platform that serves both the advanced needs of cybersecurity professionals and the critical accessibility needs of users with disabilities is a unique accomplishment we are very proud of.
  • End-to-End Automated Data Pipeline: The fully automated flow—from live-streaming a session to storing the recording in GCS and generating an AI summary via a serverless Cloud Function—is a complete and polished feature that adds immense value.

What we learned

This project was a deep dive into building a modern, scalable, AI-driven application. We gained extensive hands-on experience with:

  • Google Kubernetes Engine (GKE): We learned the intricacies of dynamic resource provisioning, RBAC security, and complex networking for a multi-tenant application.
  • Google's Agent Development Kit (ADK): We moved beyond basic examples to build a sophisticated, hierarchical system, mastering concepts like PlanReActPlanner and custom tool integration.
  • Model Context Protocol (MCP): We learned how to design a truly decoupled system where new tools and capabilities can be added without modifying the core agent logic.
  • Full-Stack Development: This project required us to work across the entire stack, from Python for AI, to Node.js for cloud orchestration, to shell scripting and Docker for containerization, all automated with GitHub Actions.

What's next for Spectra

Spectra has a solid foundation, and we are excited about its future potential:

  • Advanced AI Perception: Integrate multimodal LLMs that can directly process the visual stream from the browser, allowing the AI to "see" and interact with non-standard UI elements like canvas-based applications.
  • Sophisticated Network Interception: Fully implement the AI-powered network interception capability, allowing the agent to modify requests and responses in real-time for deeper security testing.
  • Reinforcement Learning for Browser Tasks: Explore using reinforcement learning to train the AI agent, allowing it to learn and improve its strategies for common browsing tasks or vulnerability discovery patterns over time.
  • Enhanced Agent Collaboration: Enable more complex, cooperative workflows where agents can hand off tasks to one another, further mimicking a human team.

Built With

Share this project:

Updates

Submission history