Assign to Ship

AI engineers on probation: GitLab Duo agents that earn the right to merge and deploy, one clean production release at a time, and lose it on a rollback.

Comment

AI engineers on probation. A crew of GitLab Duo agents that earn the right to ship.

Path: A, Start Fresh · Autonomy: Assisted, Supervised and Hands-off, decided per crew member by track record · Stages: all nine

The idea

The hackathon asks you to pick a level of autonomy. We think nobody should pick it once and walk away. You would not give a new hire production keys on day one, and you would not give them to an agent either. So in Assign to Ship, autonomy is earned:

Clearance What a crew member may do alone
Assisted plan, build, get CI green, approve. A human merges and releases.
Supervised all of that, plus merge. Production waits for one human click.
Hands-off merge and release to production.

Three clean production releases in a row: promoted one level. One rollback: demoted one level. The ledger (governance/clearance.json) is written only by CI, from what actually happened in production. The agents can read their own record and cannot change it.

The crew is Ada (supervised: three releases shipped, promoted once, demoted once) and Tunde (assisted, newest on the crew). You assign either one an issue in GitLab, and the live floor shows them carrying it room by room through the nine DevSecOps stages: https://web-production-ef88c.up.railway.app/shift

What one assignment does

Each crew member is a custom GitLab Duo flow with five agents and its own service account:

  1. Planner reads the issue and governance/policy.json, posts a plan, refuses out-of-policy work.
  2. Developer writes the code and tests, runs them, commits to agent/<crew>/issue-<n>, opens the MR.
  3. Verifier waits on the MR pipeline; on failure reads the failing job logs and pushes a fix (max 2).
  4. Gatekeeper checks paths, tests, the policy gate, the linked issue and the diff, then approves (label + a committed review record in reviews/) or holds the MR for a human.
  5. Releaser follows the merge through the main pipeline and posts the change record, the production URL and the clearance result on the issue, or opens an incident.

Then CI applies the crew member's clearance:

  • auto-merge merges the exact reviewed sha, for supervised and hands-off crew only.
  • deploy:production is automatic for hands-off, a human click otherwise.
  • monitor:production checks production; rollback:production rolls back on failure.
  • clearance-ledger records the outcome: streak, promotion or demotion.

Nine stages

Stage Where
Plan planner agent: plan note, labels, policy refusal
Create developer agent: code + tests, branch, MR
Verify node:test + JUnit; verifier agent fixes failures from job logs
Package non-root container image to the GitLab registry; CycloneDX SBOM
Secure GitLab SAST, secret detection, container scanning
Govern policy gate over every scanner report; gatekeeper agent; review records; clearance enforced by CI
Configure railway.json + deploy/env/*.json as code, checked before any deploy
Release versioned GitLab Release per merge; staging, then production
Monitor staging validation, production smoke + metrics, automatic rollback, scheduled synthetic check, incidents, clearance ledger

What happened when we ran it

All of this is in the project history, and the live floor and overview pages read it from the GitLab API.

  • Issue to production in about 20 minutes. #5 (add XOF) was planned, built, scanned, reviewed, merged and released to production by the agents and the pipeline.
  • A promotion, earned. Ada shipped #6 on supervised clearance. Production waited for one human click, passed its checks, and it was her third clean release in a row: Clearance: Ada clean v0.1.45, promoted supervised → hands-off.
  • A new hire, held. Tunde did the same work on #7 and his gatekeeper approved it, but on assisted clearance CI does not merge for him: !6 is waiting for a human.
  • A demotion, automatic. In a fire drill, Ada shipped #8 on hands-off clearance, straight to production with nobody watching. The production check failed, rollback:production put the previous release back, and the ledger demoted her: Clearance: Ada rollback v0.1.50, demoted hands-off → supervised. No human touched the pipeline.

What we learned by running it

Three platform rules shaped the design, and we found each one by running it:

  1. Flow triggers ignore events caused by a flow's own service account. An agent-opened MR's pipeline never woke our separate gatekeeper flow (we proved it by starting the same pipeline as a human: it fired at once). So each crew flow carries its ticket the whole way.
  2. AI workflows may not merge. /merge from a flow returns "Quick actions merge cannot be used with AI workflows". We agree with that rule. That's why the agent approves and CI merges, pinned to the reviewed commit, and clearance decides whether CI may.
  3. A release after an agent merge is still attributed to the agent, so release flows never trigger on it either. The crew member's own releaser follows it to production instead.

Our very first MR was held by the gatekeeper because it could not read the linked issue. It refused to approve a change it could not check against the request. That is exactly the behaviour probation is built on.

Built With

Share this project:

Updates

Submission history