Paper Trail

Ships merge requests to production with no person in the loop, and files a sealed, auditor-ready change record mapped to SOC 2 CC8.1 and ISO 27001 for every change.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Paper Trail

Inspiration

Every SOC 2 or ISO 27001 audit asks the same questions about every production change: who asked for it, who approved it, was it tested and scanned, when did it ship, how was it checked afterward, and how would you undo it. Small teams answer by digging through tickets and CI logs the week before the audit.

Automation makes this worse in one specific way. Once a pipeline ships changes by itself, there is no human approval to point at, so teams add a manual approval step back in just to have something to show. I wanted the opposite: keep the loop automated, and make the automation produce better evidence than the manual step ever did.

What it does

Paper Trail is a GitLab pipeline plus a small set of Duo agents that take a merge request to production and file the evidence as they go.

  1. Plan. plan:risk scores the real diff against a policy file in the repo (policy/risk-policy.yml): sensitive paths, dependency changes, infra changes, size. The score picks the gates. High risk means two review agents, a 10 / 50 / 100 percent rollout and a 15 minute watch. Low risk means one reviewer, a straight rollout and a 2 minute watch.
  2. Verify. Tests run with a JUnit report. Two GitLab Duo agents review the diff in CI through the Duo CLI, independently: one for correctness against the issue's acceptance criteria, one for risk and rollback safety. Neither wrote the change, and neither sees the other's verdict.
  3. Secure. GitLab SAST, secret detection and container scanning run on every merge request; secure:triage sends anything over the policy limit to a Duo agent that fixes it or accepts it with a written reason.
  4. Gate. gate:pre-merge checks tests, unaccepted findings and review verdicts against the risk level, and writes its decision down. On our first real run the correctness agent asked for changes because the merge request had no "Closes #N" and no acceptance criteria. The gate failed. We fixed the merge request, not the policy, and it passed.
  5. Package, configure, deploy, monitor. The image is built once and pinned by digest. Cloud Run gets a new revision with no traffic, then traffic moves in the steps the risk level demands. Each step is probed for its share of the watch window (error rate and p95 latency limits from the policy). A breach sends traffic back to the last good revision on its own.
  6. Govern. Both gate decisions, the rollout steps, the watch result and a rollback drill go into one JSON evidence pack. Every artifact is hashed, the pack is sealed with one SHA-256, and each section is mapped to the exact words of SOC 2 CC8.1 and ISO/IEC 27001:2022 Annex A 8.32.

The live page at the Cloud Run URL is that pack, made readable: risk score, the two Duo verdicts, both gates, the canary steps, the watch window, the rollback drill, a strip of GitLab's nine stages showing what each one left on file, the control map, and a download of the sealed pack.

The problem and who it is for

Engineering leads at small SaaS companies going through SOC 2 or ISO 27001 without a release engineer, and the auditors who review their changes. Change management (CC8.1) is one of the controls auditors sample most, one change at a time. Paper Trail makes each change carry its own answer.

How GitLab automates the post-code lifecycle here

Stage What runs GitLab features
Plan Risk scored from the real diff; gates chosen from policy as code CI/CD, dotenv artifacts, issues
Create Merge request that closes the issue and lists acceptance criteria with tests Merge requests, MR templates, AGENTS.md
Verify Tests; two independent Duo review agents GitLab Duo Agent Platform (Duo CLI headless in CI), JUnit reports
Package Container image built in CI; image pinned by digest for deploy Container registry, CI/CD
Secure SAST, secret detection, container scanning; Duo triage over limits GitLab security scanner templates, Duo agent
Release Release tag with agent-written notes on the default branch Releases (job written, not yet run: see below)
Configure Cloud Run revision with no traffic; OpenTofu for the service OpenTofu with GitLab-managed state
Monitor Canary steps with probes, automatic traffic rollback Environments, Cloud Run
Govern Gate decisions, hashed artifacts, sealed pack, control map Reusable CI/CD component template, CODEOWNERS

What is real today, and what is still planned

I want judges to be able to check every claim, so here is the line.

Running now, in public pipeline history:

  • Merge request pipelines with risk scoring, tests, SAST, secret detection, container scanning, two Duo review agents and the pre-merge gate. The gate has failed for a real reason (missing "Closes #N") and passed after the fix.
  • The live Cloud Run service, deployed through the canary script with the gates from the MR's risk decision: 10, 50, then 100 percent, a 900 second watch (180 probes across three 5 minute steps, 0 errors, worst-step p95 142.6 ms against a 500 ms limit), a rollback drill to the previous revision and back in 17 seconds, and both gates passing. Revision paper-trail-00003-cig, image digest sha256:d47b414fd940...
  • The sealed evidence pack and the change file page at the live URL.

Recorded from a workstation, not yet from CI: the deploy evidence for the live change came from deploy/record.sh, which runs the same scripts/rollout.sh the deploy:production job uses, because the Google Cloud key is not yet a CI/CD variable. The pack says this in its provenance section.

Written but not yet exercised end to end: the Duo intake flow that turns an issue into a merge request (.gitlab/duo/flows/paper-trail-intake.yaml), the autofix agent for failing tests, auto-merge (off by default and never on drafts), the default-branch release, OpenTofu configure and production deploy jobs, and a real rollback with an incident issue. The "rolled back" file on the live page is built from a test fixture and labeled as an example.

How we built it

Python standard library for the app and the evidence tooling (risk scoring, gates, monitor, evidence pack, HTML audit record), with 52 tests and ruff. GitLab CI/CD YAML for the pipeline, Flow Registry v1 YAML for the Duo flow, Bash for the Cloud Run rollout and rollback, OpenTofu for infrastructure, python:3.12-alpine for the image.

I built it by directing AI coding agents: I set the design, the policy and the rules (no human gates, never loosen the review policy, evidence for every step), and the agents wrote most of the code and ran the pipelines. I reviewed the results and I own all of it. The Duo agents inside the pipeline run on GitLab's models.

Challenges

  • Trigger limits. Duo flow triggers fire on actions by a person, and the later steps are done by a service account, so the review and triage agents run as CI jobs through the Duo CLI instead of as chained flows.
  • Unrelated histories. The project started with a template commit, so there was no merge base. Risk now scores the whole tree when there is no merge base, which over-scores instead of hiding changes. That is why every run so far is high risk.
  • Cloud Run reserves /healthz. Google's front end answers it before the app does, so the probe path is /health on Cloud Run.
  • An honest gate failure. The first time the Duo agents ran, the correctness agent blocked the merge for missing change evidence. Leaving the policy alone and fixing the merge request was the point of the project.
  • A review that could not see the test. Later, the correctness agent flagged a leak test as meaningless. The diff it was handed showed the fixture's secret-looking variable as [REDACTED], so the check looked vacuous. We renamed the variable so the agent could see the sentinel, tightened the test, and left the review policy alone. The next pipeline passed.

Accomplishments

  • Two independent Duo agents gating a real merge request, with their verdicts in the evidence.
  • A canary rollout whose steps and watch window come from the risk score, and a post-deploy gate that checks they matched policy.
  • One sealed file per change that an auditor can verify with one command.

What we learned

Auditors do not need a person to click approve; they need to see who or what decided, on what basis, and that the record was not edited. Agents can provide all three if every decision is written down at the moment it is made.

What's next

  • Run the full loop from the default branch in CI (key-based deploy, release, configure) and record a real rollback with an incident issue.
  • Publish the evidence step as a CI/CD Catalog component.
  • Add more control mappings (SOC 2 CC7.1, ISO A.8.9).

Testing instructions for judges

  1. Open the live URL. The page is the evidence pack for the change that is running. Use the stage strip to walk the nine stages; switch to the examples to see a rolled-back change.
  2. Download the pack from the page, then run python -m paper_trail verify --pack paper-trail-evidence-pack.json in a clone of the repo to check the seal.
  3. Open the merge request's pipelines to see the Duo review jobs and the gate.
  4. Run the tests: pip install -r requirements-dev.txt && python -m pytest -q.

Built With

Share this project:

Updates

Submission history