Inspiration
A security warning is the start of a decision. Developers still need to understand whether it affects their project, what the evidence proves, and what to investigate next.
We built VulnCouncil around a question: what if AI security reviewers could challenge each other’s conclusions before presenting advice to a developer?
What it does
VulnCouncil checks a real project’s dependency files against public vulnerability databases, then brings together three AI reviewers:
Dependency investigator: finds known security warnings. Advisory verifier: checks whether the warnings apply to the declared package versions. Remediation reviewer: examines possible fixes and identifies what still needs testing.
The reviewers compare their conclusions and challenge unsupported claims. The dashboard shows their positions, source links, review history, and remaining uncertainty.
We audited our own project’s 302 declared package versions and found a public advisory affecting its NLTK dependency. That gave us a real maintainer question to investigate. The finding establishes an affected dependency version; it does not establish that our application is exploitable.
How we built it
We built the backend with Python and FastAPI and the dashboard with React and TypeScript. Independently addressed agent workers exchange review messages through AGNTCY SLIM, with Qwen3 running locally through Ollama providing model-assisted reviews.
The auditor reads dependency files without installing packages or executing repository code. It checks OSV and GitHub advisory records, examines published releases through PyPI and npm, and preserves links and file hashes so results can be traced to their sources.
Models interpret the evidence, while code controls tool permissions, validates evidence references, and enforces limits. We also built a controlled lab for testing disagreement, changing conditions, and deterministic evidence rules.
Challenges we faced
The biggest challenge was making collaboration useful. Three fluent explanations do not automatically produce a better review. We gave each specialist a different responsibility and retained initial positions alongside peer challenges so their contributions could be inspected.
Another challenge was preventing conclusions from outrunning the evidence. An affected dependency does not prove an exploitable application, and a published upgrade does not guarantee a compatible or secure fix. In one recorded public-repository audit, a proposed upgrade still returned advisory matches. The system preserved those warnings rather than recommending it as safe.
Live data and local model inference also introduced delays and failures. We made incomplete checks, unavailable sources, and coverage limits visible, and kept saved audits clearly distinguishable from fresh runs.
What we learned
We learned that useful AI security assistance needs traceable evidence and explicit uncertainty. Disagreement can help a developer understand which questions remain unresolved.
We also learned to separate testing claims carefully. Our synthetic lab tests the deliberation protocol; real repository audits validate a different workflow. Neither establishes superiority over existing scanners.
Accomplishments
- Built a working auditor for supported Python and npm dependency files.
- Completed two real repository audits using public APIs, local model inference, and SLIM communication.
- Passed 89 tests, the production build, and desktop/mobile browser checks.
- Produced exportable reports containing findings, sources, reviewer positions, and coverage limits.
What’s next
We want to compare VulnCouncil fairly against a conventional scanner plus a single AI reviewer, test it on more repositories, and investigate whether affected components are actually used by an application.
Our goal is to help developers move from a security warning to a decision they can explain and act on.
Log in or sign up for Devpost to join the conversation.