Inspiration
Security teams have to decide which vulnerabilities to patch first. Every CVE is public in NVD, with a description and a CVSS score. But reading records one by one to find the ones that matter is slow, and in security, time is what you have least of.
What it does
Ask in plain English, for example "Which WordPress plugin vulnerabilities should we patch first?". CVE Triage Agent searches NVD records, ranks the matches by risk (actively exploited first, then CVSS score), and replies with a short, prioritized list. For a product that isn't in the data, it says so instead of guessing. Every answer shows what it cost to generate, and ends with the data snapshot date and a link to NVD.
How I built it
- Model: NVIDIA Nemotron 3 Nano on Nebius Token Factory. Every answer is at least two Nemotron calls: one uses native tool calling to plan the search (query, number of results, product filter), the other writes the answer from the ranked results.
- Agent: a framework-free agent loop and tool registry in plain Python, so every step is visible and tested.
- Search: a hybrid (BM25 + dense) retrieval service over a 5,000-record sample of 2024 NVD CVEs, hosted on Render.
- Ranking in code: the tool drops results that don't mention the product asked about, then sorts the rest by exploitation status and CVSS score before the model sees them.
- UI and hosting: Gradio, in Docker, on Render.
One rule is behind most decisions: a prompt is a request, code is a guarantee.
Challenges I ran into
- The search service always returns its top results, even for a product that isn't in the data. Without a filter, the model could present an unrelated CVE as an answer. The tool now drops results that don't mention the product.
- Letting the agent retry "Apache" as "httpd" matched unrelated products: a router's
httpdcomponent and Jasperhttpdx. I removed the retry on purpose: an honest "not found" is safer than a wrong match. - In the live demo, the model reordered the results the tool had ranked. Ranking in code only holds up to the tool output, so the answers are checked in code too.
- The full 2024 NVD set didn't fit in the memory of the free hosting tier, so the search covers a 5,000-record sample.
Accomplishments that I'm proud of
- A plain-Python evaluation with no LLM judge: five deterministic checks over six labelled questions, with 28 unit tests for the checks themselves.
- Zero made-up CVE IDs across the six questions, and an honest "not found" for all three out-of-dataset questions.
- The eval also caught real problems, and I documented them instead of hiding them: non-breaking hyphens in CVE IDs in 4 of 6 answers, and a skipped product filter in 2 of 5 product questions.
- About $0.0006 per question, cheap enough to run as a public demo.
What I learned
- What agent frameworks do under the hood, by writing the loop and the registry myself.
- An eval can be wrong too. The first run found two false alarms in my own checks, and the tool's own "not found" message can't be the ground truth, because the tool is part of the system being tested.
- Precision over recall: in security, a wrong answer can send a team to patch the wrong thing.
Log in or sign up for Devpost to join the conversation.