ECHO

• 

  Am I Safe? on GitLab Pages: version 1.8.0 maps to line 1.x and shows the release that fixes each advisory.

• 

  Live 1.x line on GitLab Pages, running path-to-regexp 1.8.2 with the ReDoS fix from ECHO.

• 

  Version Coverage Matrix, built by CI from the append-only ledger: state, gates and release for every line.

• 

  Backport MR !35 to release/3.x, opened by the Fan-out flow and merged after all gates passed.

• 

  Fix MR !13 on main: Probe Smith adds a black-box probe commit and labels the fix ECHO-2026-003.

• 

  Exploit Lab: one ReDoS request against every release line. Vulnerable builds hang past the guard; ECHO-patched builds answer in 2-15 ms.

Inspiration

A fix on main does not protect people on older versions. Someone must backport it by hand, one release line at a time. That work is slow, and the backport is rarely tested as hard as the original fix. Users on old lines stay exposed after the fix is public.

GitLab's own maintenance policy calls backporting "a high development, quality assurance, and support cost". GitLab maintains several release lines at once. Most open-source maintainers cannot afford that discipline. We built ECHO to give it to them, hands-off.

Writing the fix is not the hard part. Getting it onto every supported version, with proof, is.

What it does

A developer merges one fix on main. That is the only human step. ECHO does the rest:

1. Probe Smith (Duo flow, trigger: MR created) writes a black-box probe for the bug. CI proves that the probe fails before the fix and passes after it.
2. After merge, CI cartographs the bug. It runs the probe against the release lines and tags (53 tags across 8 release lines). It records which versions are affected and where the bug was introduced. It does not guess from version ranges.
3. Fan-out (Duo flow, supervisor plus one line_transplanter sub-agent per line) ports the intent of the fix to each diverged line. It opens one backport MR per line.
4. Each backport must pass five deterministic gates and an exploit gate. Each gate is a separate CI job:
   • G1 Reproduce, G2 Fix, G3 Suite, G4 Surface, G5 Minimality, EXPLOIT (latency limit).
   • G4 compares behaviour on a 10,000-input corpus.
   • A port that ECHO cannot prove is escalated with a one-sentence reason. It is never merged.
5. Merge Keeper merges a port only after all its gates pass. Each line then gets a patch release: npm package, container image and GitLab Release. Each line also gets a GitLab Pages deploy behind an in-CI A/B canary. The canary is promoted or rolled back automatically.
6. Ledger Keeper (Duo flow) writes an append-only ledger branch and one receipt issue per fix. Consumer Upgrade (Duo flow) opens upgrade MRs in downstream apps.

The product is live on GitLab Pages: https://showcase-e259e7.gitlab.io/

• Am I Safe? Type your version, for example 1.8.0. You get every advisory for that line and the release that fixes it.
• Version Coverage Matrix: every fix, every line, gate results and release links.
• A JSON status API (/api/status.json), a badge (/badge.svg), one live page per release line and an Exploit Lab.
• In GitLab Duo Agentic Chat you can ask "is 1.8.0 safe?". Duo answers through ECHO's MCP server.

How we built it

We forked pillarjs/path-to-regexp (MIT). It has four supported lines (6.x, 3.x, 1.x, 0.1.x) and four end-of-life lines. The upstream library code is unchanged, except for fixes merged through MRs.

GitLab does all the work:

• Five custom Duo Agent Platform flows, installed and wired through the AI Catalog API:
  • Probe Smith (trigger: MR created).
  • Fan-out (pipeline events; supervisor mode with a line_transplanter sub-agent per line).
  • Ledger Keeper (pipeline events; receipt issue and the echo-ledger branch).
  • Merge Keeper (pipeline events on backport MRs; checks the merge rules and merges if the platform allows it).
  • Consumer Upgrade (installed in the two consumer projects).
• Composite-identity service accounts and .gitlab/duo/agent-config.yml.
• One pipeline definition for every branch, merge request pipelines, a needs DAG and resource_group locking.
• Gates G1–G5 and EXPLOIT as separate CI jobs, plus SAST and Secret Detection.
• npm Package Registry, Container Registry and the generic package registry.
• GitLab Releases through the release: keyword.
• Environments line/<id> and line/<id>/canary.
• GitLab Pages for the per-line deploys, the matrix, the status API and the badge.
• An OpenTofu stack on GitLab-managed Terraform state.
• An MCP server, so Duo Chat can query ECHO's ledger.

The agents write code. The gates decide. No agent can push its own work past a gate.

Challenges we ran into

• Flows cannot merge. Flow identities get HTTP 403 insufficient_scope from the merge API. Our project role is Developer, so we cannot store a write token in CI variables. The Merge Keeper flow runs the merge checks in GitLab and records the 403. The actual merges are done by an external Merge Keeper bot. It applies the same F-10 checks as the CI merge guard and runs with the operator's token.
• No Google Cloud. We had no billing account. The Cloud Run and OpenTofu GCP code is in the repo, but we did not use it. Deploys run on GitLab Pages with an in-CI A/B canary.
• The AI probe was too narrow once. For the ReDoS fix, the probe from Probe Smith tried only the literal separator. It missed the escaped-separator variant on 3.x. ECHO flagged the gap against ground truth (the maintainers' real backports). We left that evidence unchanged and report it as a finding.
• Ports that included gate files. G5 rejected the first ReDoS ports because they committed gate artifacts. ECHO escalated them. Recovery needed an operator requeue, so that run took 88m 41s from merge to the last release.
• We list every operator action in our runs in DEVPOST.md in the repo.

Accomplishments that we're proud of

ECHO caught a real gap in the maintainers' own fix. We replayed a real 2024 CVE (CVE-2024-45296, ReDoS). ECHO's gates rejected the maintainers' historical 0.1.10 backport. After their patch, the exploit still took 3,569.5 ms against a 300 ms limit. That line really was still vulnerable. It was later disclosed as CVE-2024-52798 and fixed only in 0.1.12. ECHO's own 0.1.x port takes the same input from 3,877.6 ms to 0.3 ms.

Measured results (GitLab API and the ECHO ledger):

• Run 2: the fix MR merged at 10:25 UTC. The last of the four releases shipped at 10:56 UTC. The Merge Keeper bot merged each backport only after all its gates passed.
• Three more fixes (ECHO-2026-004, -005, -006) also reached every affected line, with each backport merged only after its gates passed: 14 ECHO releases in total. Two of them needed a manual Fan-out start after three merges landed within a minute and their main release jobs raced for the same version; the release guard caught it.
• Rollback drill: a faulty canary got 40/40 errors. ECHO rolled it back automatically.
• Downstream apps: both consumer apps upgraded automatically (consumer-router-app to 6.2.4 and then 6.2.5, consumer-legacy-api to 1.8.2).
• Replays: two fresh replay projects produced identical cartography from scratch.

What we learned

• Let agents write and let deterministic gates decide. The gates caught the narrow probe and the weak 0.1.10 backport. A reviewer reading the diff would likely miss both.
• Cherry-pick fails when release lines diverge. Porting the intent of a fix works, but only with a gate that checks behaviour, not text.
• Ground truth is worth the effort. Comparing ECHO to the maintainers' real backports showed us where ECHO is strong and where it is weak.

What's next

• Make Probe Smith cover every variant in an advisory, so the 3.x miss cannot happen again.
• Let the Merge Keeper flow merge directly when flow tokens get merge scope.
• Add adapters for more libraries and ecosystems beyond npm.
• Turn on the Cloud Run deploy path when billing is available.

Built With

• gitlab
• gitlab-ci
• gitlab-duo
• gitlab-pages
• mcp
• node.js
• opentofu
• path-to-regexp
• python
• typescript