Upstream Intake

GitLab Duo agents take each new release of an open-source app into our patched fork, fix what breaks, and ask one person to approve.

Comment

How to check it

  • https://linkding.aii.ai.kr/health shows the deployed build (1.47.0+fork.<commit>). It is served from a Compute Engine VM on Google Cloud (34.132.55.46); see deploy/gcp/.
  • Merge requests !1 to !4 are the real releases, !5 to !7 the drills and !8 a validation run, each with the agents' notes and commits.
  • Agent session pages need a GitLab login; the README shows two of them as screenshots.

Inspiration

Many teams that self-host an open-source tool end up patching it: a session policy, a header their intranet needs, an audit log for the security team. From then on every upstream release is a chore. Someone has to read the diff, put the patches back, find out what broke and deploy. So it waits, and upstream's security fixes wait with it.

Taking a release is also the moment a supply-chain problem walks in. The xz-utils backdoor came in through release tarballs and binary test files. Elasticsearch, Terraform and Redis changed their licenses from one release to the next. We wanted every release read, and the routine work done, before anyone is asked to decide.

Tools exist for parts of this. Renovate and Dependabot open merge requests for dependency bumps, and fork-sync bots merge upstream as it is. None of them reads what a release does to the patches you carry, retires a patch that upstream made unnecessary, or proves with a test that a patch still works.

What it does

We forked linkding, a self-hosted bookmark manager (MIT, 11k stars), at v1.45.0 and added three patches a company would plausibly carry: 8-hour sessions, a crawler header for intranet pages, and a JSON audit log. Each patch has a contract test that states what users rely on. Then we took in the four real upstream releases that followed, v1.46.0 to v1.47.0. Upstream published them between August 9 and September 13; we replayed them in order on 2026-10-16, starting the watch step by hand after each merge (the daily schedule runs the same command).

For each release:

  1. The intake watch merges the release into an intake/<tag> branch and runs a deterministic risk scan of upstream's diff: license files and metadata, build and install scripts, binary test files, encoded payloads, new dependencies, first-time contributors in sensitive code, overlap with our patches. It opens an issue and a merge request.
  2. Creating the merge request triggers the reviewer flow. It opens the flagged upstream commits, explains each one, and decides per patch: carry, port or retire. Its second agent, the porter, makes those changes in one commit.
  3. The pipeline runs a govern gate, upstream's tests (1,300 to 1,380), our contract tests, SAST, secret detection and a job that separates new findings from findings already on main. Nothing from the release is built or tested before the gate passes. The image goes to staging and a check confirms staging serves that exact build.
  4. If the pipeline fails, the fixer flow finds the upstream commit behind the failure and repairs the fork, preferring a setting over a code change. If it passes, the reporter flow writes "Ready for your decision".
  5. A person reads the summary and merges. Production deploys from a registry tag, is checked, watched for ten minutes and rolled back if it fails.

What happened on the real releases:

  • v1.46.0 (75 commits, conflict in settings): the reviewer found that upstream #1295 adds a session length setting, so our session patch could go, and that commit 0c2362b renames the session cookie, which signs every user out once. The porter retired the patch, moved our 8 hours to LD_SESSION_COOKIE_AGE and updated the contract test. The reporter refused to call the release scanned because no SAST job had run. It was right; we fixed the pipeline.
  • v1.46.1 and v1.46.2: clean merges, patches carried, 0 new security findings.
  • v1.47.0: upstream added SSRF protection. The reviewer saw that our intranet bookmarks would lose their titles; the porter allowed .corp.internal with one setting (in production only; a follow-up commit copied it to staging, and the porter's instructions now keep both files in step). A new secret-scan hit in a test file stopped the pipeline until the merge request was labelled as reviewed. The reporter asked for a look at the themes after the switch to esbuild; staging and production screenshots showed no visible difference.

Three drills:

  • A fake release that switches the license to source-available terms and adds an npm postinstall that pipes a download into a shell: the gate stopped it before any of it was built. The reviewer said stop.
  • A refactor of upstream's HTTP client that silently drops our header and still passes upstream's tests: the reviewer caught it and the porter kept our header.
  • The same release with the reviewer switched off: our contract test failed, and the fixer named the upstream commit and repaired http_client.request().

After hardening the pipeline (intake tools run from main, protected files, required scanner reports) we ran the second drill again as a validation run. It found two bugs in our new checks and one gap: a job that failed during a GitLab outage would have gone to the fixer. All three are fixed, with tests.

How we built it

  • GitLab Duo Agent Platform: three custom flows in the AI Catalog (Flow Registry v1, API-only tools such as get_commit_diff, get_repository_file, create_commit, get_job_logs and note tools). The reviewer has a "merge request created" trigger. The fixer and reporter are started through the Flows API. Each flow runs as its own service account, so every commit and note shows which agent made it.
  • GitLab CI/CD: conflict check, govern gate, tests, contract tests, SAST, secret detection, security diff, image build with registry cache, staging and production deploys by registry tag, verification, ten-minute monitor, rollback, GitLab release.
  • Intake tools in Python (standard library and PyYAML), with tests on synthetic git histories, including an xz-style case.
  • Deploys are pull-based: a Compute Engine VM on Google Cloud (e2-micro, us-central1) follows the staging and production tags in the GitLab container registry. The pipeline stores no secrets and no cloud credentials: registry pushes use GitLab's job token, and the VM pulls tags. The gcloud commands and the VM setup are in deploy/gcp/.

Challenges we ran into

  • Our hackathon role could not create CI/CD variables, schedule variables or project access tokens. We moved the two steps that need a token (opening the intake merge request, starting a flow) into a small controller and kept the pipeline free of secrets.
  • GitLab only lets people fire flow triggers, so a pipeline started by an agent's commit does not start the next agent. The controller's follow-up step looks at each finished intake pipeline and starts the fixer or reporter once.
  • Security scans on an intake branch report everything upstream ever had. We added a job that marks which findings the release adds. On v1.46.1 it reported 15 findings, none of them new.

Accomplishments that we're proud of

  • Four real upstream releases taken in and deployed. From intake merge request to merge took 38, 11, 9 and 133 minutes; the long one waited for a review of a new secret-scan hit. Each release needed one decision, the merge. Production now runs upstream v1.47.0 with two of the original three patches.
  • From the upstream diffs alone, without test results, the reviewer named three problems: a renamed session cookie, a patch that upstream made unnecessary, and intranet titles lost to the new SSRF protection.
  • The reporter would not claim a security scan that had not happened.

What we learned

  • Contract tests let us check what the agents changed. They say what must stay true; the agents can change everything else.
  • A deterministic scan that points the reviewer at specific commits worked better for us than asking a model to read a diff of several thousand lines.
  • Asking the summary agent to cite only what it read kept it honest.

What's next

  • Read the release tarball, not only the git tag, since that is where xz hid.
  • Take several releases per intake for forks far behind upstream.
  • A group-level setup so several forked apps share one set of intake flows.

Path B: what we added or changed during the submission period

Everything except upstream linkding's own code, all made between 2026-10-15 and 2026-10-24 (UTC), after the October 5 start: the fork itself (from upstream v1.45.0), the three patches and their contract tests, the patch ledger, the intake tools, the GitLab pipeline, the three Duo flows, the deploy setup on Google Cloud, the intakes of v1.46.0 to v1.47.0 and the drills.

Before (how a patched fork is usually kept up): upstream linkding builds and tests with GitHub Actions. A patched copy has nothing for taking new releases. The merge, the review, the patch porting and the deployment are manual.

After: each release arrives as one merge request with a risk scan, an agent review, ported patches, tests, security scans, a staging deploy and a summary. A person reads the summary and merges; production follows.

Built With

Share this project:

Updates

Submission history