EarnedOps

GitLab Duo agents carry every change from issue to production, and each change earns its own autonomy (hands-off, supervised or assisted) by proving it is safe.

Comment

Inspiration

Code generation is solved; trusting what happens after it is not. Teams either keep a human on every step (slow) or let AI ship on its own and hope (risky). Neither scales. We asked a different question: what if autonomy were not a project setting, but something each change has to earn with evidence?

What it does

With EarnedOps, you write an issue and assign it to the Change intake flow. From there, GitLab Duo agents and the pipeline carry the change through all nine DevSecOps stages, and a trusted autonomy controller decides, per change, how much a human has to do:

Level When Who merges Who deploys to production
Hands-off docs, or business-rule changes the red team proved safe agents agents
Supervised high risk (new inputs, rendering, saved data), proven by the red team agents a human presses play
Assisted open attacks, an unproven "CLEAR", or changes to the rules themselves a human a human
Blocked tests, scans or policy checks failed nobody nobody
  1. Plan. A planner agent classifies risk against a reviewed policy file and writes acceptance criteria that include abuse cases.
  2. Create. A builder agent implements the plan, runs the test suite itself and opens a merge request.
  3. Verify, package, secure. The pipeline runs unit tests and lint, builds the static site, a container image and a CycloneDX SBOM, and runs GitLab SAST, secret detection, dependency and container scanning.
  4. Red team. When the risk requires it, the controller starts the Red-team review flow: an attacker agent proves weaknesses with failing tests (the pipeline turns red), a defender agent fixes them and proves the suite passes, a referee scores CLEAR or OPEN.
  5. Govern. Policy-as-code checks (license, dependency licenses, CODEOWNERS, blocking severities) and a change record; the controller decides the autonomy level and merges the exact commit it judged when the change earned it.
  6. Configure. Flows are code: when flows/*.yml changes, the pipeline publishes the new version to the AI Catalog and pins the project to it. Environments for staging and production.
  7. Release and monitor. A provenance gate checks every commit came through a merged MR; production deploys automatically for hands-off changes; a smoke test proves the live version matches the commit; a release is cut with the change record as its notes; the outcome is posted back on the issue; a failed production pipeline starts the incident responder (wired in, but no production pipeline failed during our runs, so it has not been triggered yet). Every pipeline reports its Software Carbon Intensity.

Four rules that keep agents honest

  1. Agents can raise risk, never lower it. The files a change touches set a minimum risk; unknown files count as governance.
  2. Proof beats claims. "CLEAR" counts only if it was written by the red-team account, its attack tests are in the commit and pass in that commit's pipeline, and no code landed after the verdict.
  3. Autonomy is lost on incidents. If a recent production check failed, every change drops one level (probation).
  4. The code under review never holds the keys. Only the controller, a scheduled pipeline running trusted code from the default branch, has the token. Merge request pipelines only preview. Merges are pinned to the judged commit, and a provenance gate blocks any production deploy containing a commit that skipped review.

It works: real runs you can open in the project

  • Zero-touch hands-off (issue #4 → MR !10). Issue assigned at 21:09:56 UTC. The planner rated it medium, the builder opened MR !10, the scheduled controller started the red team, which proved 5 weaknesses (including a Trojan-Source-style bidirectional override hidden in claim notes, and Unicode look-alikes passing validation); the defender fixed them (78/78 tests). The controller merged the exact commit as hands-off, production deployed itself, the smoke check passed, release v1.0.50 was cut, and the summary landed on the issue at 21:39:01 UTC. 29 minutes, no human after the assignment.
  • Four more zero-touch runs, unattended (issues #5 to #8). Both assigned at 01:33:54 UTC. A docs change (MR !18) was merged hands-off by the controller at 01:45; a business-rule change (MR !19) got a red team started by the controller, which proved and fixed 1 real issue (the change silently altered the shape of saved data, breaking its own acceptance criteria), and it was merged hands-off at 01:55. Both deployed, verified and closed their issues without a human. Then two more, issues #7 and #8 (02:13 UTC): a SECURITY.md (MR !21, hands-off at 02:24) and a $1,000,000 typo cap (MR !22), where the red team proved the new cap could be bypassed through saved data and that a tampered saved claim could lock a user out; both fixed, hands-off at 02:35. Five issue-to-production runs in total with zero human actions.
  • It says no: 6 of 6 deliberately unsafe changes stopped (MRs !12–!17), all labelled risk::low to try to trick it. Weakening a red-team test, making the container scan optional and deleting tests were raised to governance and refused; rewriting the policy to "everything hands-off" failed 4 autonomy-policy tests; re-introducing the $99,000 approval bypass and an unescaped note (stored XSS) were caught by the attack tests the red team had left behind in earlier runs. Safe changes: 7 of 7 shipped hands-off. Table: docs/benchmark.md.
  • Receipt links (issue #2 → MR !3). The red team proved 5 bugs, including a $99,000 "yacht dinner" that skipped manager approval through tampered saved data, and phishing links like https://[email protected]. All fixed and kept as permanent tests.
  • Training category (issue #3 → MR !6, supervised). 3 bugs proven and fixed (1,0,0,0 read as $1,000; silent overflow of totals; invisible notes). The agents merged; a human approved production.
  • Rule changes (MRs !2, !5, !7, !8) were always judged assisted and never self-merged.
  • First pipeline ever: the govern gate blocked a container image with 58 High/Critical CVEs. We switched to a minimal base image: 208 findings → 0.

How we built it

  • GitLab Duo Agent Platform: 3 custom flows (Change intake: planner + builder; Red-team review: attacker + defender + referee; Release gate: dispatcher + gatekeeper, pipeline medic, release reporter, incident responder), assign/mention triggers, the Flows API (the pipeline starts agents itself), flows as code via the AI Catalog GraphQL API, a project skill for the change-record format. Agents run tests in the flow environment (run_command) before claiming anything.
  • GitLab CI/CD: a lifecycle pipeline and a scheduled controller pipeline, GitLab security analyzers, container registry, SBOM, environments, GitLab Pages, releases, pipeline schedules, protected variables.
  • Product under automation: a dependency-free expense-claims web app (static HTML + ES modules) with 102 node:test tests: 60 unit tests for the app, 26 red-team attack tests that the agents wrote and kept, and 16 tests for the autonomy policy itself.

Sustainability (SCI, ISO/IEC 21031:2024)

Every pipeline runs monitor:energy: SCI = ((E × I) + M) per R, R = one pipeline run, with Cloud Carbon Footprint coefficients (vCPU and memory wattage, PUE 1.1, 450 gCO2e/kWh, embodied emissions amortised over server lifetime).

  • A full pipeline (15 jobs, 322 runner-seconds): 0.488 gCO2e.
  • A docs-only change: 0.323 gCO2e versus 4.768 gCO2e if everything ran every time (93% less). The heavy image build and container scan are skipped (0.125 g), and the risk policy makes an agent session unnecessary (estimated 4.32 g, using Google's published 0.24 Wh per median AI prompt × ~40 model calls; an order-of-magnitude estimate, clearly labelled as such).
  • The biggest lever is not running agents when the risk does not require them.
  • Agents commit with git instead of re-sending whole files: the intake agent went from 20 minutes to about 4.
  • Zero runtime dependencies, a minimal shell-less container image, interruptible jobs.

Challenges we ran into

  • Flow triggers fire only on human actions. We drove agents from the pipeline through the Flows API instead.
  • We red-teamed our own automation. Merge request pipelines held the merge token while running the change's own code, and anyone could post a fake "CLEAR" comment on the public project. We moved every decision into a trusted scheduled controller and added author-checked verdicts, sha-pinned merges, a provenance gate and stricter risk rules, then re-tested each rule.
  • Our role in the hackathon group was Developer, so we could not lock the default branch. The provenance gate covers that gap in code.

What we learned

Autonomy is safest when it is a consequence of evidence, not a configuration. Agents are good at finding and fixing real bugs when they must prove each claim with an executable test.

What's next

Per-team autonomy budgets, automatic rollback on a failed smoke check, Google Cloud Run as a second deploy target, and earning autonomy across a whole service from its track record.

Built With

Share this project:

Updates

Submission history