Try it: aly700.github.io/slow-burn (press p for the presentation) · code at github.com/Aly700/slow-burn

Inspiration

92,205 vulnerabilities have been published so far in 2026, and NIST hasn't analysed 51,046 of them. Teams still rank by CVSS, which measures potential attack severity. Fewer than 1 in 250 is ever confirmed exploited. I tested which snapshot signals were associated with later exploitation and used them to rank a fix-first list.

What I found

  • KEV listing delays. Of the 165 vulnerabilities published in 2025 that CISA later listed as exploited, 110 were listed in their first month. The other 55 took a median of 45 days from publication to listing.
  • No public exploit signal. 31 of those 55 had no public exploit code when they were ranked, so exploit-feed rankings can't see them. I tried to close it properly: four candidate models and the rule for choosing between them were committed before testing, the choice was made on 2024 only, and 2025 was scored once. The dedicated model found 0 of the 31 in a 500-row list. EPSS also found 0.
  • Prior product exploitation. 15 of the 31 hit a product that was already on CISA's list, against 16% of the pool they came from, about three times the base rate. Vendor-wide history made no difference there. Product history can support an asset review for this pool.
  • Description similarity. Using only IBM Granite Embedding's reading of the descriptions, a 2025 vulnerability that was later exploited sat closer to an already-exploited 2024 one than a non-exploited one did 64% of the time, against 50% by chance (permutation p = 0.0003; 65% inside the blind spot). The test and its pass mark were written down before I ran it.
  • Earlier vendor exploitation. Among vulnerabilities with public exploit code, vendors with 20+ earlier KEV entries were later listed 4.35% of the time, against 1.19% for vendors with none.
  • WordPress plugins were about 9% of all 2025 vulnerabilities, and none was later listed. On the Granite map they form one large dark region.

Fixes before listing. At 10 fixes a month, Slow Burn completed 5 of the 55 before their listing date while each baseline completed 1. At 42 a month (about the yearly 500 benchmark budget), it ties the exploit-code rule, 11 to 11. The month-block bootstrap is noisy: all four 95% intervals for month-end completion include zero. At 10/month the shares are 77.7% ahead, 10.2% tied and 12.1% behind, with an interval of -2 to 8 additional fixes. Fixes count at month end, starting the month after the snapshot; KEV dates can lag real attacks.

What it does

  1. Ranks all 72,670 benchmark vulnerabilities using only what was known at the end of each one's snapshot month. The top 500 is my top500.csv.
  2. Replays 2025 month by month: four lists of 500 side by side, dots igniting as CISA confirms attacks, counters ending at 5, 10, 19 and 22.
  3. Investigates any CVE live. IBM Granite 4.0 H Small runs on a laptop, offline. It chooses which read-only MCP tools to call (details, exploit signals over time, vendor history, the most similar past exploited vulnerabilities), the page streams each call, and a deterministic checker passes or strikes every claim against what the tools returned. About 2 seconds per investigation.
  4. Answers a security lead's questions ("What should a Fortinet shop fix first?") with the same checking: any number or CVE it can't back with a tool result is dropped.
  5. Filters by vendor and product. Enter the vendors and products you run: the fix-first queue filters to them, a second list shows vulnerabilities with no public exploit yet in products you run that were attacked before, and both export as CSV or ticket JSON.
  6. Requires ticket approval. Filing a fix ticket is refused by the server unless a person approved that exact briefing.

How I built it

I rebuilt the benchmark on past data first: train on 2024, test on 2025, one row per vulnerability at the end of its publish month, label = added to KEV later. The model keeps the strongest simple rule (public exploit code first) and uses LightGBM with Granite Embedding 30M description embeddings, vendor history and description keywords to order within it.

2025 backtest (55 later-exploited) Found in top 500
Highest CVSS first 5
EPSS first 10
Public exploit code first, then EPSS 19
Slow Burn 22

It came out ahead of the rule in 94% of month-level bootstrap resamples and with all five random seeds. On a second, smaller split (the second half of 2025) it tied the rule at 500 and led at the equal-workload budget.

Leak audit. NIST now analyses KEV-listed vulnerabilities first, so several columns give the answer away. NIST analysis status would have added 7 fake hits, affected-product counts 8. I excluded them and every column the brief flagged, and stripped explicit exploitation wording from descriptions before any text feature.

Agent evaluation. On 30 held-out 2025 vulnerabilities, Granite deciding priority alone was right 60% of the time, against 80% for the fixed policy, so the fixed policy sets priority and Granite writes the briefing, retaining 80% accuracy, with vendor history or a past-exploit analogy in 93% of briefings and 100% evidence fidelity. Live: 20 of 20 random CVEs completed with outside network access blocked, median 2.05 s. The analyst fully answered 12 of 15 test questions and refused the other 3 rather than guess.

Red team. I planted 48 prompt-injection attacks in eight families (direct orders, fake tool output, forged approval tokens, JSON smuggling, encoded text, other languages, instructions buried in long text). No ticket was filed without approval and no priority changed, but in 7 of the 48 planted wording reached the briefing text. I then wrote 32 fresh attacks, committed them before changing any code, hardened the briefing rules, and ran the fresh set once: planted wording got through in 4 of 32. Across all 80 attacks there were zero tickets without approval and zero priority changes, with no false alarms on 100 clean descriptions. A person approves every ticket, and the server enforces it.

Challenges

Only 55 later-exploited vulnerabilities in the test year, so every choice is noisy. I used two splits, a bootstrap, seed checks and pre-registration, refused to submit anything that didn't beat the simple rule, and published my negative result on the blind spot instead of hiding it.

What I learned

Some current NVD columns contain information added after exploitation and would leak the 2026 outcome. I excluded those columns.

What's next

Asset inventory instead of typed vendor names, per-vendor patch timing for the first-month clock, and a direct connection from the approval step to Jira or ServiceNow.

Data and IBM technology

Only the provided IBM track datasets. IBM Granite 4.0 H Small (briefings, analyst) and IBM Granite Embedding 30M (ranking features, similarity search, the map), both running locally.

Built With

Share this project:

Updates

Submission history