Inspiration

Security scanners can produce more findings than a small team can investigate. CVSS helps describe technical severity, but it does not say whether a vulnerability is being exploited or whether the affected asset is exposed. MegaPatch explores how a small team maintaining Python services could decide what to review first.

This is a product hypothesis, not a validated customer need: we have not interviewed teams or measured the ranking against a labeled dataset.

What it does

MegaPatch combines security triage with evidence a person can inspect:

Prioritizes dependency advisories. ScoreRisk is a deterministic rule—not an AI decision—that combines exploitation evidence (EPSS and CISA KEV), exposure declared by the user, and CVSS. Missing inputs remain unknown and can produce a score range.
Keeps code findings separate in a local scan. When scanning your own repository, Bandit findings appear in their own queue, ordered by Bandit severity and confidence; they are not assigned a ScoreRisk score. The demo's ranking of the practice app does give its code findings a ScoreRisk score.
Verifies findings on a practice app. MegaPatch includes a deliberately vulnerable Flask app and PoCs for SQL injection, OS command injection, and path traversal. These PoCs target only that bundled app, never a scanned repository, public URL, or external IP.
Proposes a narrow patch. For a compatible Bandit B602 finding, Nemotron can propose a change. Before sending, the local interface shows the exact function text and asks for consent. A proposal can pass static checks and a person can approve and export its diff, but the model-written code is not run or tested against the application in this version.

The public demo is read-only and shows recorded runs. It does not scan visitors’ repositories or run new model-written patches.

What the recorded runs show

The recorded runs from October 10 and 12 predate the current rule that disables execution of model-written code. In those runs, the same PoC and functional tests judged each patch in a Nebius Sandbox. One failed patch was rejected, sent back to Nemotron once, and repaired. In another run, a person declined a path-traversal patch that an earlier judge had accepted; a stricter re-judge later rejected it. The original records were preserved.

Two runs used --yes, which skipped approval questions; no person approved patches in those runs. In the two runs without --yes, a person decided on each patch.

The demo’s ranking example uses real scanner output from nine deliberately outdated pinned packages, plus findings from MegaPatch’s practice app and one manual finding. It contains 72 distinct findings and 62 CVEs; one CVE is in CISA KEV, and 59 have EPSS below 0.05. The asset exposure is declared against a fictional inventory, not a customer system. For example, a CVSS 9.8 certifi advisory ranks below a lower-CVSS Pillow advisory in KEV and marked internet-facing. Across 216 tested weight and constant combinations, that ordering remained stable. These results are a demonstration, not a measure of real-world ranking precision.

How we built it

Python and Flask implement the practice app, PoCs, and functional tests.
Streamlit provides the dashboard and the local repository-scanning interface.
Bandit and pip-audit provide static code and dependency findings. Repository scanning is read-only: it does not run the scanned project. Dependency and threat-data queries are optional; by default, nothing is sent externally.
Nebius Token Factory serves NVIDIA Nemotron 3 Super through its OpenAI-compatible API.
Nebius Token Factory Sandboxes run the PoCs on the practice app in the pipeline. In the recorded runs they also ran model-written patches; the current version does not run those patches.
Tavily is an optional source of public evidence for findings that cannot be verified by a PoC; it does not affect ScoreRisk.
Run records are stored as JSON. python judge_check.py checks their consistency and runs the bundled PoCs and functional tests offline.

The project was built with AI coding assistance (Claude Code).

Challenges we ran into

A judge is only as strong as its PoCs and tests. A path-traversal patch passed an earlier, weaker check because the exploit did not cover a sibling-directory edge case. A person caught the problem, so the PoC and judge were strengthened. A separate recorded patch also failed functional tests, was rejected, and was repaired using the judge’s output.

We also had to decide whether to keep running model-written code. In a separate check on October 10, code ran as root inside a Nebius Sandbox, and one outbound TCP connection from that sandbox to pypi.org:443 succeeded. That observation does not establish the full extent of the sandbox’s network access, but it means we could not demonstrate that it was offline. The current version therefore keeps model-written code as a proposal. It does not execute it locally or in the sandbox.

Accomplishments

Recorded the full exploit–patch–retest flow on the bundled practice app before disabling model-patch execution.
Preserved the run records, including a rejected-and-repaired patch and a later stricter re-judge.
Built an offline check so judges can inspect the practice-app PoCs, functional tests, and consistency of the recorded evidence without our API keys.
Added a local, read-only Python repository scan and a consent-gated, statically reviewed patch proposal for one finding type.

What we learned

The test used to judge a patch matters as much as the model that writes it.
A sandbox label alone does not prove that outbound network access is blocked.
Honest limits make it clearer what the evidence demonstrates—and what it does not.

Limitations and next steps

MegaPatch currently verifies only three vulnerability classes, and only against its own practice app. Its local repository scan does not execute or test a team’s application. The demo ranking uses a deliberately outdated package set and fictional exposure declarations; we have not measured ranking precision or validated the product with customers.

Before enabling execution of model-written patches again, the project needs a sandbox operation with networking disabled and a check inside that same operation before any candidate runs. Next, we would test the ranking with a small set of teams and labeled findings, with their consent.