ProofOps Agent Swarm

• 

  ProofOps+™: one-click agentic security verification—Attack → Patch → Replay with auditable proof.

• 

  Auto Pilot scores Shield Integrity, Disruption Index, Coverage & Time-to-Adapt across test packs.

• 

  Auto Pilot scores Shield Integrity, Disruption Index, Coverage & Time-to-Adapt across test packs.

• 

  Choose from multiple attack types—prompt, tool misuse, vision, audio, and video—to test real multimodal risk.

• 

  Active Modalities show what’s being exercised: TXT, TOOL, IMG, AUD, and VID guardrails.

• 

  Reproduce the issue on demand: ProofOps runs the attack and logs the exact failure.

• 

  Apply a targeted policy patch to block/redact the unsafe behavior—no guesswork.

• 

  Replay verifies the fix: the same attack is blocked/redacted and the run returns to green.

• 

  Timeline captures every step for auditability—what happened, when, and in what order.

• 

  Findings summarize exploit paths and severity, turning raw failures into prioritized actions.

• 

  Policy Diff shows what changed between runs—clear, reviewable remediation deltas.

• 

  Ledger Export produces proof artifacts you can share—repeatable evidence of security posture.

Inspiration

Software supply‑chain attacks and misconfigured AI pipelines pose an existential risk to open‑source projects. Volunteer maintainers spend countless hours triaging bug bounty reports, searching for secrets and updating compliance docs instead of building features. We envisioned an agentic swarm that automates this verification loop and produces evidence that any third party can trust.

What it does

ProofOps+ Agent Swarm uses a state‑machine orchestrator to coordinate specialized Gemini 3 agents. Red‑team agents perform static analysis, secret scanning and prompt‑injection tests across code, container images and test data. Blue‑team agents reproduce issues in isolated sandboxes, write failing tests and generate patch proposals. A judge agent scores findings based on severity and reproducibility and packages the run into a cryptographically signed Proof Ledger. The latest build introduces audio and video jailbreak packs alongside text and image scenarios. Dedicated processors send media to Gemini’s multimodal API; the HUD lights up “AUD” and “VID” badges to show active modalities and the system refuses hidden instructions in speech or video. A live scorecard reports shield integrity, disruption index, family coverage and time‑to‑adapt, and an interactive timeline shows failures and replays. After patching, the orchestrator re‑runs the failed scenarios to prove the fix and updates policies automatically.

How we built it

The project is composed of a Python/Node.js backend and a React frontend. We use Google Cloud Run to host the orchestrator, agents and multimodal processors, and Firestore to store runs and ledger files. The attack‑pack library is versioned YAML and includes families like Secrets, Prompt Injection, RAG, Loop Injection, Tool Misuse, PII, Visual Jailbreak and Audio/Video Jailbreak. For multimodal processing, we implemented audio_processor.py and video_processor.py modules that stream media to Gemini 3 Pro via Google AI Studio and perform transcription and frame analysis. The Blue‑team agent uses Gemini’s structured output to propose JSON‑schema patches, and the HUD is built with Chart.js and tailwind for responsive metrics. An Antigravity IDE plugin allows us to prototype scenarios and edit policies directly from the dashboard. The entire loop runs deterministically under a referee that enforces allow‑list policies and budgets.

Challenges we ran into

• Balancing Gemini’s API rate limits and budgets while streaming audio and video; we batch assets and cache results.
• Designing prompts that keep agents focused on policy‑compliant behaviour despite adversarial multimodal inputs.
• Coordinating asynchronous agents without cross‑talk; we built a reputation system to allocate tasks to the most capable agent.
• Ensuring reproducibility: ledger signatures, replay tokens and stable IDs required careful serialization and hashing.

Accomplishments that we’re proud of

• Deployed a working prototype that can audit a public GitHub repository, run eight threat families with text, image, audio and video attacks, and maintain 100 % shield integrity.
• Implemented audio and video processors that successfully identify hidden instructions in speech and video and update the UI in real time.
• Created a deterministic referee that scores evidence objectively and packages it into a signed Proof Ledger with transcripts.
• Reduced triage time by automating bug bounty report generation and PII redaction; the new run patches issues within ~2 seconds.
• Integrated Google AI Studio and Antigravity to demonstrate how Gemini 3’s multimodal reasoning enables features beyond chat.

What we learned

• Structured prompts and JSON‑schema outputs enable reliable agent coordination and patch proposals.
• Multimodal attacks (audio/video) are easy to overlook; dedicated processors and visual indicators are essential.
• Cryptographically verifiable evidence builds trust and can become a standard for software security attestation.
• Deploying on Cloud Run with environment‑level secrets simplifies scaling and reduces overhead.

What’s next

• Extend the attack pack library with cross‑modal reasoning scenarios (e.g., combining diagrams and speech).
• Add image generation and annotation features in the HUD, showing patched diffs and evidence snapshots.
• Conduct a pilot on a popular open‑source project and publish metrics on vulnerabilities found, time saved and reproducibility.
• Release our Antigravity plugin and attack‑pack authoring tools to the community for contributions.

Built With

• chartjs
• github-api
• google-ai-studio
• google-cloud-functions
• google-cloud-run
• google-gemini-3
• node.js
• postgresql
• python
• react