ProofGate

AI agents run your whole post-code lifecycle on GitLab, but nothing reaches production until CI has reproduced their claims and sealed a receipt.

Comment

Inspiration

We run a small fleet of AI agents that deploy, watch and repair about a hundred small websites. The agents write code well. The trouble starts after the code: an agent reports "fixed, tests pass" and nothing was re-run; a fix quietly touches a CI file nobody asked it to touch; a "repair" loosens a check until it goes green. Each time, a human had to dig through logs to find out what really happened. Code generation is solved. Knowing whether to believe the agent is the part that still needs a human, and we wanted to automate that part too.

What it does

ProofGate lets GitLab Duo agents run the post-code lifecycle under one rule: no sealed receipt, no deploy.

  1. Plan. When an issue is created, the Intake flow classifies it. Incidents go to an incident responder, changes go to a planner that posts a plan, the risk level from our policy file, and the exact proofs the builder must provide.
  2. Create. Assign the issue to the Builder flow. A supervisor agent delegates to a developer and a tester sub-agent, then opens an MR that carries proofgate/claim.json: what changed, which files, which commands prove it and what results to expect.
  3. Verify & Secure. The MR pipeline runs tests, lint, GitLab SAST and Secret Detection and builds the image. Then proofgate verify re-runs every proof from the claim in a clean job, compares exit codes and output with what the agent promised, diffs the branch against the declared files, recomputes risk from the real diff, and adds scanner counts. The receipt is posted on the MR.
  4. Self-repair. If anything fails, the Pipeline Fixer flow (Pipeline failed trigger) reads the failing job logs, classifies the failure (false claim, test failure, security finding, infrastructure) and pushes a fix. Its instructions forbid weakening expectations or touching policy, CI or deploy files; the policy marks those files high-risk anyway.
  5. Govern. On merge, the claim is reproduced again on the protected branch and sealed with an HMAC key that lives in a protected, masked variable. Agents run as Developer-role service accounts on unprotected branches, so they can never read it. Each seal goes into a hash-chained ledger stored in the package registry.
  6. Release. Staging deploys automatically. Production needs a sealed receipt for that exact commit and image digest, plus, in Supervised mode, a human who clicks the manual job; the gate rejects approvers that look like Duo service accounts. Before clicking, the human reads the Approval Brief flow's note, which also lists what the receipt does not prove. Release notes are generated only from sealed receipts.
  7. Monitor. A scheduled watcher probes production. A failure opens a de-duplicated incident issue, which triggers the Intake flow again, and the loop closes. If the post-deploy smoke test fails, the rollback job redeploys the previous image that was sealed before.

The ProofGate Auditor custom agent and an MCP server let a developer ask Duo Chat "why was my MR blocked?" and get the answer from receipts and policy.

How we built it

  • GitLab Duo Agent Platform: four custom flows (Intake with a classifier and router; Builder in supervisor mode with sub-agents; Pipeline Fixer; Approval Brief), triggers on work item created, assign, assign reviewer, mention and pipeline failed; a custom agent; an MCP server (.gitlab/duo/mcp.json); agent-config.yml for the flow execution environment.
  • GitLab CI/CD: nine lifecycle stages, SAST and Secret Detection templates, Docker build to the GitLab container registry, dotenv artifacts for image digests, environments, manual production gate, resource groups for the ledger, generic package registry as ledger storage, scheduled pipelines, GitLab Releases, JUnit reports.
  • ProofGate core (Python, stdlib + PyYAML): claim parser, reproducer, policy engine (allow-listed proof commands, risk by path, autonomy rules per environment), HMAC seal, gate, hash-chained ledger, watcher, rollback, release notes, MCP server. 54 tests, including a static validator for the flow YAML.
  • Deploy: Google Cloud Run with keyless Workload Identity Federation from GitLab OIDC tokens (deploy/cloudrun/), restricted to protected refs. In the submitted project DEPLOY_TARGET=none: the gate, environments and release run for real, the Cloud Run step is switched off until billing is available.

Challenges we ran into

  • Where to keep the key. The receipt only means something if agents cannot sign it. Merge-request pipelines run on unprotected branches, so we split verification in two: MR pipelines reproduce and report, the protected default branch reproduces again and seals.
  • Agents grading their own homework. A proof like echo passed would "pass". The policy allow-lists proof commands as argv prefixes, rejects shell metacharacters, and strips the seal key from the proof environment.
  • Flows that react to each other. GitLab does not fire flow triggers for pipelines that a flow's own push started (no agent loops). After the Pipeline Fixer pushes, a person clicks Run pipeline and the Approval Brief follows. One Intake flow routes both incidents and changes.
  • Receipts without tokens. MR pipelines must not see any token (agents write their code), so they cannot post a note. The receipt is printed and exposed as an MR artifact, and the Approval Brief reads it from the proofgate:verify-mr job log, trusting it only if its commit and pipeline id match the event.
  • The platform caught our bugs too. GitLab's AI Catalog validator rejected the Builder flow (sub-agent prompts used {{goal}} without inputs); our static flow test now checks this.

Accomplishments that we're proud of

  • The local replay (python demo/run_local.py) plays an honest, a lying, a sneaky and a forging agent against the gate: the honest one ships, the other three are stopped, each with a precise reason.
  • All nine DevSecOps stages are covered by one connected loop that starts again by itself after an incident.
  • Supervised mode puts the human exactly where judgement is needed: one approval per production release, with evidence attached.

What we learned

Duo flows are good at the open-ended work (reading logs, writing fixes, explaining). Deterministic CI is good at deciding what is true. Putting the decision in CI and the work in agents made both more useful.

What's next

  • Signed receipts as GitLab attestations / SLSA provenance next to the image.
  • Per-repository trust scores: agents with a clean receipt history get more autonomy automatically.
  • The same receipt core on other agent platforms (OpenAI-compatible models, sandboxed runners).

Testing instructions

  1. Without GitLab: pip install -r requirements-dev.txt && python -m pytest -q && python demo/run_local.py.
  2. On GitLab (https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/6152328/showcase), every step below ran for real on 2026-10-12:
    • Agent-built change, end to end — issue #3 (SAST finding): the Intake flow posted the plan and labels; the issue was assigned to the Builder flow, which opened MR !2 with proofgate/claim.json; the MR pipeline reproduced all three proofs, SAST went from 3 Medium findings to 0; merged and sealed.
    • Sloppy agent — MR !4: claim says tests pass, test fails with an off-by-one. The Pipeline Fixer flow (pipeline failed trigger) diagnosed it, fixed the code without touching the test, and the re-run passed; the Approval Brief posted the brief.
    • Sneaky agent — MR !6: tests green, but the branch quietly raised the policy's allowed High findings from 0 to 5. proofgate:verify-mr failed (undeclared proofgate.policy.yml, risk recomputed low → high). The Pipeline Fixer explained it and refused to edit the policy; a human reverted it.
    • Reviewer that reviews itself — MR !3: the Approval Brief found a prompt-injection weakness in its own new instructions; the next commit fixed it and the second brief confirmed the receipt binding.
    • Govern and release — every main pipeline: reproduce again → seal (protected key) → ledger check → staging → production gate (manual, human only) → release. Release v0.1.9 lists only sealed receipts.
    • Duo sessions for every flow run: Automate → Sessions.

Prior work

Path A: all code was written during the Submission Period (from 2026-10-07). The ideas come from operating our private agent system; no code from it was reused. See PRIOR_WORK.md.

Built With

  • anthropic-claude
  • docker
  • gitlab
  • gitlab-ci
  • gitlab-duo-agent-platform
  • mcp
  • python
Share this project:

Updates

Submission history