repo.spot

An autonomous DevSecOps squad of 3 Qwen-powered agents — Auditor, Architect, QA — that triage repo vulnerabilities, propose patches, and approve fixes in a constrained, verifiable loop.

Comment

Inspiration

Security vulnerabilities in open-source repositories are found, triaged, and fixed by humans — a slow, expensive, and error-prone process. As a developer who maintains multiple open-source projects and actively tracks the AI/ML ecosystem, I wanted to explore whether a disciplined multi-agent system could replace that manual review loop with something faster, more consistent, and fully auditable.

The Agent Society track's emphasis on measurable gains over a single agent made this the perfect frame: not one "smart" model, but a structured team of three agents with distinct roles, hard stopping rules, and verifiable outputs.

What it does

repo.spot is an autonomous DevSecOps squad powered by Qwen Cloud. Given a repository URL or a set of files, it spins up a constrained 3-agent review pipeline:

  • Auditor — scans the codebase, identifies vulnerabilities, and ranks each finding by a calculated risk score (severity × exploitability × exposure). Only findings above a confidence threshold of 0.65 proceed.
  • Architect — receives the Auditor's ranked findings and proposes the minimal safe patch for each, outputting a structured diff with reasoning and assumptions listed explicitly.
  • QA — reviews the proposed patch against the original finding. It either APPROVEs, requests a REVISE with one concrete blocking reason, or ESCALATEs to human review.

The loop runs for a maximum of 2 rounds per file. If QA is not satisfied after round 2, the finding is escalated rather than left in an unresolved state. Every agent response is validated against a strict JSON schema before the workflow continues — invalid responses trigger one automatic repair prompt, then escalate on second failure.

The entire agent dialogue is streamed to a live frontend so judges and users can watch the swarm reason in real time.

How I built it

The orchestrator is written in TypeScript and communicates with all three agents via Qwen Cloud's Model Studio API. Each agent is a stateless function: it receives a typed JSON envelope, returns a typed JSON response, and never holds conversational state between rounds.

Key architectural decisions:

  • Model-agnostic config — agents are selected via environment variables (MODEL_AUDITOR, MODEL_ARCHITECT, MODEL_QA) so the best available Qwen endpoint can be swapped without touching orchestration logic.
  • Hard termination rules — MAX_ROUNDS=2 and RISK_THRESHOLD=0.65 are enforced at the orchestrator level, not inside agent prompts, preventing any agent from extending the loop.
  • Schema-first design — every inter-agent message conforms to a shared Zod schema (agentSchemas.ts), making the system trivially testable and the demo replay-safe.
  • Streaming UI — the orchestrator pipes agent log events to a lightweight frontend via server-sent events, giving a real-time view of triage → patch → challenge → approve/escalate.

Hosted and inferred entirely on Qwen Cloud (Alibaba Cloud), using the hackathon credits for compute and API access.

Challenges I ran into

The hardest problem was preventing the agents from entering "philosophical negotiation" — open-ended back-and-forth that looks impressive in a demo but fails under deadline pressure and produces no deterministic output. The fix was architectural: move all termination logic into the orchestrator, not the agent prompts, so no single agent can extend its own life.

A second challenge was schema reliability. Early iterations saw agents occasionally producing malformed JSON when patch diffs were large. The one-retry-then-escalate pattern solved this without sacrificing demo stability.

Accomplishments that I'm proud of

  • A fully working 3-agent loop that produces structured, human-reviewable patch candidates with zero hallucinated file paths in testing.
  • A real-time streaming UI that makes agent reasoning visible and verifiable — not a black box.
  • A clean separation between orchestration logic and agent prompts, making the system easy to extend with a fourth agent (e.g. a Risk Prioritiser) without rewriting the core loop.

What I learned

Bounded agents outperform free agents in production demos. Giving each agent a single, narrow job and a hard exit condition produces more reliable and more impressive results than giving agents broad autonomy and hoping they converge.

Qwen Cloud's Model Studio API is genuinely fast at multi-turn structured completions, and the latency per round is low enough that the streaming UI feels responsive rather than sluggish.

What's next for repo.spot

  • A fourth agent: Risk Prioritiser — sits before the Auditor and filters incoming findings by business impact, so the squad focuses on what matters most first.
  • GitHub Actions integration — trigger the squad automatically on every PR, post findings as review comments, and block merge on unresolved escalations.
  • Federation layer (A2M) — integrate with A2M (a local-first packaging and auditing layer for AI agents) so repo.spot's agent manifests can be shared, versioned, and verified across teams.

Built With

Share this project:

Updates