Lockkeeper

• 

• 

• 

• 

• 

• 

Inspiration

Code generation got fast, but what happens to code after it ships did not. We measured it on linkding, a well-kept open-source bookmark manager. Over three years, a fix for one of its dependencies took a median of 45 days to land on main. On 56% of days, at least one fixable high-severity advisory was still open. That is not negligence; it is what one person's spare time allows. And since 11 September 2026, the EU Cyber Resilience Act gives any manufacturer selling in the EU 24 hours to report a vulnerability that is being actively exploited.

We wanted agents that close that gap, and that we could trust to act alone: agents that stop acting alone the moment they get something wrong.

What it does

Lockkeeper watches what actually runs in production (28 of linkding's 56 locked packages) against OSV, CISA KEV and EPSS. For each upgrade it opens a case issue with a deadline, and then:

1. Separates risk from noise. A read-only Duo agent claims where each flaw is reached, and a parser checks every claim against the code. On linkding it found 21 advisories and one real exposure: urllib3, because bookmark URLs come from users and are streamed. PyJWT is off in this deployment (OIDC is disabled), and Django and Soup Sieve are never reached.
2. Fixes and proves it. uv bumps the lockfile; a model does not. An agent writes a proof-of-fix test that must fail on the vulnerable version and pass on the fix. For urllib3, the test the agent wrote fails on 2.7.0 and passes on 2.8.0 in CI. A separate gatekeeper agent reviews the change. Its verdict counts only if it comes from the gatekeeper's own service account and is newer than the head commit.
3. Decides in code how much it may do alone. Each change class has a ceiling in lockkeeper.toml. Within that ceiling the level is earned: clean outcomes promote a class, and one rollback drops it to Assisted. Changes people made in the project's history earn at most Supervised. Hands-off needs Lockkeeper's own outcomes, from replays of past fixes and from real production results. Missing evidence caps the level, and a kill switch stops everything.
4. Ships it through Cloud Run staging and a 10% canary with synthetic probes, then promotes it or rolls back automatically.
5. Publishes the evidence:
   • a GitLab release with a CycloneDX SBOM and OpenVEX;
   • a public status page;
   • drafts of the Article 14 reports, with a live clock. A named person submits them, because ENISA's platform has no API.
6. Bills its own carbon for every fix (ISO/IEC 21031 SCI, including the Duo sessions), verified with the Green Software Foundation's Impact Framework.

What happened on the real project (9 October 2026). One scan opened four cases:

• Django (L2) merged 6 minutes after its case opened.
• PyJWT (L3; all 14 advisories gated) merged unattended after 18 minutes.
• Soup Sieve (L3, not reachable) merged unattended after 46 minutes.
• urllib3 (L2: 2 reachable, 1 honest "unknown") merged after 90 minutes. Because of the "unknown", production waited for a person.

After approval, the canary passed its probes (30 of 30 samples, no errors). All four fixes went to production in one deployment.

Then we broke a canary on purpose. We deployed a revision whose container exits at once. It never became ready and never received a request, and production stayed up throughout. The outcome was recorded as rolled-back on drill case #7. On the status page, dependency-patch dropped from L3 Hands-off to L1 Assisted, and it now needs three clean outcomes to reach L2 again.

We also rehearsed the CRA clock as a fire drill on #6. It treats urllib3's CVE-2026-97689 as actively exploited in the version that ran before the fix. Lockkeeper drafted all three Article 14 reports against their deadlines: the early warning (24 h), the notification (72 h) and the final report (14 days after the fix).

How we built it

• GitLab Duo Agent Platform.
  • Four custom flows. Remediate has an analyst and an author with separate tool sets; the others are gatekeeper, scribe and incident. The catalog-sync component publishes them to the AI Catalog from this repository, and our tests validate them against the catalog's schema.
  • A custom chat agent.
  • The Flows API, which CI uses to start the flows unattended, because flow triggers need a human actor.
  • An MCP server that exposes Lockkeeper's deterministic checks to Duo Chat and the Duo CLI.
  • AGENTS.md, chat rules and Code Review instructions.
• GitLab CI/CD. One pipeline covers all nine lifecycle stages:
  • plan: the watch and the case issues
  • create: fix branches and MRs
  • verify: tests, proof of fix and the gatekeeper
  • package: the image and SBOM
  • secure: SAST, secret detection and container scanning
  • configure: OpenTofu, with state in a versioned GCS bucket
  • release: staging, the status page and a release gate into a production child pipeline
  • monitor: canary probes, then promote or roll back
  • govern: autonomy decisions, outcomes, VEX, the GitLab release, CRA drafts and the carbon bill
• Google Cloud.
  • Cloud Run (production, staging and the status page), Artifact Registry and Secret Manager, all in europe-north1.
  • Workload Identity Federation, so no service-account keys exist.
  • Cloud Monitoring, which feeds the carbon bill.
• Python with no runtime dependencies, and 280 tests. The repository tests its own pipeline and flow definitions, and one test parses every lockkeeper command in the pipeline with the real CLI.

Challenges we ran into

• Running agents unattended. Flow triggers need a human actor, so CI drives the flows through the Flows API instead. The result is a hands-off chain that still respects GitLab's permissions.
• Trusting the right voice on a public project. Anyone can comment on a public merge request. A review counts only when the gatekeeper's own service account wrote it, and only for the commit it read.
• Testing the way back down, for real. A rollback that has never run is a hope, not a feature. So we broke a canary on purpose, in production. The drill surfaced a few edge cases in how GitLab CI and Cloud Run behave during a failure. Each is now handled and pinned by a test, and the next deployment went green end to end.
• Measuring agent carbon without token counts. Duo flows do not report tokens, so we rebuild every model turn from the session trace. That is how we found that agents dominate the bill.

Accomplishments that we're proud of

• Autonomy that is earned, revocable and explained. Every MR shows the evidence table and why the level is what it is. We revoked it for real: one deliberately broken canary took dependency-patch from Hands-off to Assisted.
• One real risk out of 21 alerts. Every verdict was checked against the code, and an agent wrote the proof-of-fix test that CI verified.
• The before and after numbers come from the same tool. Before Lockkeeper's first fix, main took a median of 45 days to adopt a fix. With Lockkeeper, fixes merged in 6 to 90 minutes and reached production the same afternoon.

What we learned

• An honest "unknown" is worth more than a confident guess. The urllib3 agent could not tell whether proxy settings expose the third advisory. It said so, and that kept a person in the loop for production, which is exactly the right outcome.
• Agents are where the carbon is. Across the four cases, the agents' share of each bill ranged from 0.6 g to 18.7 g. Measuring it changed how the agents work: the author agent now saves its evidence first, and in its next run it committed within 18 seconds.
• A drill teaches more than a design review. Failing production on purpose showed us more about our pipeline in an hour than any test on a laptop. It is now how Lockkeeper proves the way back down.

What's next

• Roll Lockkeeper out across a group of projects.
• Extend it to npm and container OS packages.
• Add signed provenance.
• File CRA reports through ENISA's API once it exists.
• Cut agent energy with fewer, cached turns.

Path B: what we added during the submission period

Everything after the commit chore: adopt linkding for Lockkeeper, all on or after 2026-10-07:

• the lockkeeper/ package and lockkeeper.toml;
• the flows and agent in .gitlab/duo/, and AGENTS.md;
• the pipeline in .gitlab-ci.yml, .gitlab/ci/ and templates/;
• infra/;
• the docs.

linkding's own code is unchanged, except pytest.ini (test paths and a marker) and the README move. Original project: sissbruecker/linkding (MIT), imported at 27b7303.

Sustainability

Every fix MR gets a carbon bill under the Software Carbon Intensity specification (ISO/IEC 21031:2024), with low and high bounds. It covers:

• CI jobs and their network traffic;
• the Duo sessions that worked on the MR (SCI for AI, Consumer);
• Cloud Run time.

Grid intensity is location-based, with no offsets. Each report writes an Impact Framework manifest, and if-run reproduces our number to one part in a billion.

Measured on the four real cases: 2.4, 6.2, 8.9 and 27.0 gCO₂e per remediated case, 44.4 g in all. Agents were the largest share in the two cases where they explored longest.

These are the design choices we measured:

• Path filtering skipped linkding's suite on 11 of 17 pushes to main. That avoided 64 job-minutes and 4.5 g.
• Everything we control runs in europe-north1, at 29.8 g/kWh against 579 g/kWh in us-east1: a nineteenth of the carbon for the same work.
• Scale-to-zero Cloud Run saved 1.1 g over 1.3 hours, compared with one warm instance per service.
• Deterministic pre-filters mean the models see the one reachable advisory, not all 21.
• "Save evidence first" turns a timed-out agent session from total waste into partial progress.

The method, boundary, uncertainty and per-case table are in docs/lockkeeper/sustainability.md.

Built With

• ai-catalog
• artifact-registry
• cisa-kev
• cloud-monitoring
• cyclonedx
• django
• docker
• epss
• gitlab
• gitlab-ci
• gitlab-duo
• gitlab-duo-agent-platform
• google-cloud
• google-cloud-run
• impact-framework
• mcp
• opentofu
• openvex
• osv
• python
• secret-manager
• uv
• workload-identity-federation