Change Passport

Inspiration

Code generation is solved; getting a change safely into production is not. Small teams in regulated UK businesses (payments, health, insurance) still need proof that every production change was reviewed, tested, scanned and approved, plus a rollback plan when it touches something risky. That proof is usually assembled by hand when an auditor asks, months later. Meanwhile AI-written merge requests pile up behind failing scans and coverage gaps that a person has to fix. I wanted the pipeline to produce the evidence and the agents to do the fixing, with a human approving outcomes rather than every step.

What it does

Each merge request pipeline produces a change passport: a JSON record of the tests that ran, diff coverage for the changed lines, security findings in the changed files, and policy rules from .passport/policy.yml (for example, changes to pricing logic need a rollback plan and a customer impact statement in the MR). It carries a sha256 digest, so any edit is detectable.

GitLab Duo flows act on it:

1. Plan. When an issue is created, a flow adds acceptance criteria, a test plan, a predicted risk label and the MR checklist the policy will demand.
2. Fix. When an MR pipeline fails, the MR flow reads the passport from the job log and routes. In the recorded run, the remediation lead had the security fixer replace an f-string SQL query (flagged by lint) with a parameterised one and the test engineer write 15 tests (changed-file coverage 100%). Those tests found a real bug: the amount limit was not enforced on the new path. It pushed one commit and listed the rollback plan and customer impact for the author, because agents never write those.
3. Publish. When the passport is green, the flow posts a plain-English evidence summary with the risk score and digest, and labels the MR passport::pass. It even flagged that the author's rollback plan no longer matched the final code. A human approves and merges.
4. Release and monitor. The main pipeline builds the image, deploys that exact image to staging on Render, runs synthetic checks (health, commit match, known quote, p95 latency), waits for a person to run the manual production job, checks again and creates a GitLab Release with the passport attached. The release flow posts a summary, or opens an incident with the monitoring evidence if checks fail.
5. Audit. The Passport Auditor agent answers "show me the evidence for this change" in Duo Chat from passports, pipelines and notes, and says plainly what it cannot prove (for example, a missing reviewer approval).

How I built it

• Passport engine (Python, src/change_passport/): parses unified diffs, GitLab security reports, JUnit and Cobertura, maps uncovered changed lines to functions with the Python AST, scores risk, renders Markdown and computes the digest. It runs as a CLI in CI and as a dependency-free MCP server.
• Flows: written against the flow registry v1 schema and checked by an offline validator (tools/validate_flows.py) for custom-flow restrictions, tool names, prompt placeholders, router graph, and the things GitLab.com's flow editor rejects (HTML comments and tags in prompts, max_cycles).
• Pipeline: .gitlab-ci.yml with SAST and Secret Detection, the passport gate, a Docker build to the GitLab registry, deploys to Render that pull the image by commit SHA (Cloud Run with keyless OIDC is also supported), and a post-deploy monitor.
• Sample app: a FastAPI service that quotes card-acceptance fees in pence for a UK payments business, plus demo patches that drive the story.
• 114 automated tests cover the engine, CLI, MCP protocol, flow validator, CI config and monitor.

Challenges I ran into

• GitLab.com's flow editor rejected my first flows: HTML comments and <placeholder> text in prompts fail its prompt security check, and max_cycles is not accepted. I added both rules to the offline validator so they fail locally first.
• GitLab does not trigger flows from a pipeline that a flow's own push started. I first saw it as a stalled loop. It is a sensible guard, so the author re-runs the pipeline after each agent commit, which keeps a human between every agent change and the next verdict.
• The agent's first fix passed tests but failed ruff format --check, so the second round failed on lint. The MR flow now formats before it commits.
• Evidence has to be trustworthy: an LLM summarising logs is not audit evidence, so the evidence is computed deterministically and the agents only act on it.

Accomplishments that I'm proud of

• A real end-to-end run on MR !1: failing passport, two agent remediation rounds, the author filling in the human-only sections, a green passport (risk 64, high) and a merge. The run is documented in docs/evidence/.
• The test agent's new tests caught a genuine bug in the "AI-written" feature.
• Diff coverage and untested-function detection tell the test agent exactly which tests to write.
• A tamper-evident passport that anyone can verify with one command.

What I learned

• Flows, triggers and service accounts are separate pieces: a flow does nothing until a trigger is enabled and its service account is a project member.
• A supervisor with narrow sub-agents worked better than one large prompt, and a one-word gate made routing dependable.
• Wherever a prompt was doing arithmetic or parsing, deterministic code did it better.

What's next

• Sign passports with Sigstore keyless signing using the same GitLab OIDC token.
• Policy packs for ISO 27001 Annex A 8.32 and SOC 2 CC8.1 change management.
• Multi-service passports and merge-train awareness.
• Publish the flows in the AI Catalog for other teams.

Built With

• docker
• fastapi
• gitlab
• gitlab-ci/cd
• gitlab-container-registry
• gitlab-duo-agent-platform
• gitlab-releases
• gitlab-sast
• gitlab-secret-detection
• model-context-protocol
• pytest
• python
• render
• ruff