ws-backport

When a security fix lands on main, the versions most people actually run are left behind. Over a fifth of ws downloads, about 41 million a week, are on major lines first released in 2018 and 2019. They get a fix only when a maintainer ports it by hand. For this CVE the ws maintainer did, ahead of disclosure; many maintainers can't, and nobody ports to lines whose support has ended. Between 0.7 and 0.9 million downloads a week (npm, rolling 7 days: 0.69M on 22 Oct, 0.92M on 27 Oct) are still on ws 3.x and 4.x, which never got this fix. GitLab does this porting for its own three maintained versions twice a month. Backport bots copy a commit and stop when it conflicts. On ws, git cherry-pick of the CVE-2024-37890 fix conflicts on every old line, and git's automatic resolution still crashes on each of them and breaks 13 to 161 tests. Of 22 real backports in ws history, plain cherry-pick conflicted on 15.

What it does

We port behaviour, not diffs. A fix is defined by what it changes: this request used to crash the server, now it gets a 400.

1. When a [security] fix merges, CI loads every supported line and end-of-life release into a server and sends the attack. A crash means affected, with no model involved. The same probe matched the advisory on all 15 published releases in its test suite.
2. CI tries a plain cherry-pick per line and posts the conflicts to a tracking issue.
3. A maintainer mentions the Duo agent: propose v6.x. A custom flow reads the old code and posts a patch with a decision and a reason for every hunk.
4. The maintainer replies approve v6.x, or revise v6.x: <feedback> to send it back. On approval, a second flow commits exactly that patch and opens the merge request.
5. On the MR, CI proves the crash is gone and that no test newly fails against the line's own baseline, and grades the patch against the real maintainer's backport when one exists. Two checks added after the first run also require the new regression test to fail on the unpatched line, and the committed changes to match the approved patch.
6. A human merges and clicks publish: a scoped npm package, a tag and a Release. A daily job installs the shipped packages, runs the exploit, and opens an incident on a regression. An approval ledger records every step and checks each approver's role against the project members list after the fact.

Lines nobody had patched (v4.x and v3.x, 27 Oct). Upstream stopped supporting ws 4.x in 2018 and 3.x in 2017 and never patched them, so no answer key exists anywhere. The agent backported both: on 3.x it found the same code under different file names (lib/WebSocketServer.js) by itself. Shipped as @aryan-group7171043/[email protected] and @3.3.4 (MRs !11 and !12). The agent adapted the fix to 4.x's different code, and every check ran live on MR !11: the MR is identical, byte for byte, to the approved patch applied to its base; the crash became a 400; no test broke; the agent's new test fails on the unpatched line. From propose to the published @aryan-group7171043/[email protected]: 15 minutes.

Result on CVE-2024-37890 (v7.x, v6.x, v5.x), with the maintainer's own backports as the answer key:

• Every agent MR passed every check that existed at the time. The one real judgement call was the client-side hunk: the agent kept it on 7.x, whose client has the same crash, and dropped it on 6.x and 5.x, whose clients never read the header. It matched the maintainer on all 3 lines (12 of 12 file-level decisions in total).
• The maintainer's own regression tests pass on the agent's patches (4 of 4), and 0 tests broke.
• The agent's 79 tool calls never read the maintainer's backports. Its code differs from the maintainer's (text match 38%, 0%, 17%).
• Two checks were added after this run merged: the approved patch applied to the base must equal the MR exactly, and the new test must fail on the unpatched line. Run on the three agent MRs afterwards, all pass.
• From the fix merging to three published, checked packages: 37 minutes, including fixing a flow bug mid-run. Ledger: 9 repository or package changes, 0 without an approval before them.

How we built it

• GitLab Duo Agent Platform: two custom flows (propose, approve) kept in the repository and registered through the AI Catalog GraphQL API, with mention triggers on per-flow service accounts. The model is used only where code can't do the job: adapting a patch to old code and explaining each hunk.
• GitLab CI/CD: the behaviour matrix, cherry-pick, test-suite delta, regression-test check, approved-patch gate and report card, all exposed on the MR widget.
• Governance: protected release branches, merge only when the pipeline succeeds, an approval ledger that checks each approver's role.
• Package and Release: the npm package registry and the release: keyword.
• Secure: one vulnerability record per release line, linked to its MR and resolved (written after the run); Semgrep SAST on branch pipelines with every finding triaged.
• Monitor: a pipeline schedule checks shipped packages and opens incidents.
• Pages: a live status page rebuilt from pipeline data.
• Integrity: the patch the agent is graded against is the maintainer's own backport. Since 25 Oct, grading jobs fetch those from GitHub, which the flow sandbox can't reach; before that they were in the project, and the exported traces show the agent never read them. One labelled copy (the maintainer's 6.x backport, MR !6) stays on purpose, to show the grading scores the maintainer's own patch at 100%.
• Evidence: every agent tool call, the later gates run on the agent's MRs, project settings, vulnerability records.

Challenges we ran into

• Old lines diverged a lot: 3.x keeps the same code under different file names, and 4.x folds the check into one combined condition. The agent has to find where the code lives before deciding anything.
• Proving the agent didn't copy. We exported every agent tool call (0 answer-key reads), moved the answer keys out of the project for grading, and added end-of-life lines where no maintainer backport exists anywhere.
• Making approval binding, not ceremonial: the approved-patch gate applies the approved patch to the MR's base and requires the MR to be identical to the result, every file, byte for byte. Any edit to the gate or the CI file blocks the merge. Only listed approvers count, and every pipeline runs the CI file and tools from master, so an agent branch can't weaken the check. On the three agent MRs, the result is identical.

Accomplishments that we're proud of

• The agent made the same call as the real maintainer on every line (3 of 3 client-hunk decisions), on a real CVE in a widely used library, with the maintainer's own tests passing.
• Checks that fail the wrong patches: a lazy patch (test only) and git's automatic resolution both fail; the maintainer's real backport passes.
• A ledger that names who approved each step and checks their role.

What we learned

The model is most useful in a narrow role. Everything a judge needs to trust (which lines are affected, whether the patch works, whether it matches what was approved) can be deterministic and re-runnable. The agent only needs to do the part code can't.

What's next

• Backport a second, larger fix, CVE-2026-48779 (a seven-file receiver change). It is already defined, validated against its fix commit, and detected on every line; the status page tracks it as awaiting a backport.
• Package the harness as a CI/CD component so any project can install it with one config file.

What's real and what's replayed. ws, its history, the CVEs, the maintainer's backports and every pipeline are real. The release branches were reset to their pre-fix versions so the 2024 fix could arrive as a new MR. Three labelled control MRs (!4 a test-only patch, !5 git's automatic merge, !6 the maintainer's real backport) show the checks fail the wrong patches and pass the right one. The shipped packages are replay artifacts and predate ws's 2026 fixes.

Limits we want judges to know. One CVE has been run through the agent, and the model may remember its public 2024 fix even though it never read it; that is why the v4.x result, where no fix exists to remember, matters. The rule that text match is style-only was set at 11:01 UTC, two minutes after the first proposal and before any MR was graded. In this run the human reviews were short (about 2 minutes per proposal) and the three merges and publish clicks came seconds apart. Anyone can comment on the public tracking issue; a planted patch still can't merge unless the approver approves exactly that patch. The v5.x–v7.x release notes were edited after publishing to add advisory links and to date the approved-patch check.

Built With

• git
• gitlab
• gitlab-ci/cd
• gitlab-duo-agent-platform
• gitlab-package-registry
• gitlab-pages
• graphql
• javascript
• node.js
• npm
• semgrep