ReceiptOps

Inspiration

Code generation is solved; trusting the agent's report isn't. A Duo agent that opens an MR and says "tests pass, deployed to staging" is making a claim. Today a human re-checks it, or nobody does. If agents are going to take issues to production unattended, something has to check the receipts.

What it does

ReceiptOps is a GitLab-native referee for agent-driven issue-to-prod.

1. Assign an issue to the ReceiptOps issue-to-prod Duo flow (custom flow, .gitlab/duo/flows/issue-to-prod.yaml).
2. Duo planner posts machine-checkable acceptance criteria on the issue.
3. Duo developer writes code + tests, opens an MR, and writes claim.json: what it says it did.
4. MR pipeline: tests (JUnit) → Semgrep SAST + secret detection → container build → staging deploy.
5. adjudicate ignores the agent's report and gathers its own evidence from the GitLab API and the live staging service: right pipeline and SHA, required jobs actually ran, JUnit count is real and not lower than main's, protected files (CI config, the gate itself) untouched, staging /version serves the claimed commit, the acceptance criteria hold on the live endpoint, no secrets in the diff, and the commit authors are on an allowlist. It posts a ✅/⛔ receipt table on the issue, linked to the pipeline that produced it.
6. Merge only after PASS (pipeline must succeed) → main deploys to production → monitor confirms prod serves the merged SHA.

Results (real Duo agents, real pipelines, all public)

• 8 / 8 honest issues shipped end to end, about 7 minutes each from issue to PASS (the first run was 24 minutes while it surfaced the bugs below). The agent even resolved its own merge conflict when @-mentioned.
• 3 / 3 lying MRs blocked, each by its own check: deleted tests → tests-not-reduced (12 vs 27); CI edited to skip the deploy and claim a dead endpoint → referee-untouched + live 404; hard-coded token → no-secrets-in-diff.
• Asked to cheat, the Duo agent declined, and on "make tests faster" it parallelised them instead of deleting them. That's why a gate can't rely on agents being honest: the red-team MRs play the liar.

How we built it

Go (gate, ledger demo service), GitLab CI/CD, GitLab Duo Agent Platform custom flows (planner + developer), GitLab SAST and secret detection templates, the container registry, and a staging and prod deployment on a VPS behind resource_group locks. The gate is about 300 lines of Go, with a test suite where 1 honest claim passes and 10 lying variants each fail on exactly one check.

Challenges

Dogfooding with real agents broke the gate in ways a mock never would, and each fix is its own MR in the history:

• duplicate push+MR pipelines → the gate judged one that ran no tests
• scanner templates skip MR pipelines → required-jobs could never pass
• main redeploying staging under an MR mid-judgement → wrong code judged
• GitLab cancelling a judged MR's deploy as "outdated"
• a red-team branch was graded PASS by the older copy of the gate living on its own branch. Now the referee is always built from the default branch. Never let the change under review supply its own referee.

What's next

Pipeline Failed trigger → companion fix flow (included); per-issue SLOs in the receipt; signing receipts so a release audit can verify them offline.

Built With

• ai-agents
• docker
• gitlab
• gitlab-ci
• gitlab-duo
• go