Release Evidence Gate

• 

  Baseline pipeline 2956668296: 21 tests and six scenarios. Commit 3899a0ef. Captured before the later Duo repair (MR !1).

• 

  Baseline CI: 21 tests, zero failures/errors at a73590f7. Later Duo repair adds the 22nd test; see pipeline 2956768550.

Inspiration

An AI agent saying a repair is ready does not answer a maintainer's practical questions: which revision was tested, where are the reports, and do they still match? Release Evidence Gate turns those assertions into a compact, inspectable evidence packet for small teams reviewing AI-generated changes.

What it does

The Python verifier checks test, compilation and narrowly scoped source reports against an expected commit. It rejects missing artifacts, altered digests, stale report identities, failed status, duplicate stages, invalid schema types and non-positive check counts. It rejects unsafe paths and symlinks within its documented stable-filesystem threat model.

GitLab CI runs verification, publishes JUnit results and retains a manifest, reports and a commit-bound decision. Six reproducible scenarios show valid evidence accepted; altered, stale, missing and failed evidence rejected; and jointly rewritten reports and manifest accepted, deliberately exposing the limit of an integrity-only check.

Actual GitLab Duo workflow

On 16 October 2026, GitLab Duo Agentic Chat read six project files and identified a real untested defect: Python treats True and 1.0 as equal to 1, so the schema version check accepted invalid types. After review of its proposed patch, Duo created branch duo/strict-schema-v1, committed a minimal two-file correction and opened merge request !1. Both the commit and MR creation passed explicit supervision prompts.

Pipeline 2956768550 passed for commit 1fbb759fcc02cb1572143a4d355fab92f4a33171. Its job log confirms the new regression test, all 22 tests, a matching decision and all six demonstration outcomes. A separate assistant-run local validation confirmed both regression subcases fail before the fix and pass afterward. This is actual Duo-assisted development with supervised approval, not a claim of autonomous deployment or external human auditing.

How we built it

Python 3.12 and the standard library implement the verifier, evidence runner and demonstration. GitLab CI provides commit identity, execution and artifacts. GitLab Duo Agentic Chat provides the code review and proposed repair. The build check compiles gate.py only; the source check detects bare eval/exec calls in that file only. Neither is a vulnerability audit or a complete package build.

Challenges and lessons

An earlier review found that unittest can report success when every case is skipped or is an expected failure. The runner now requires positive successful-check evidence, with isolated subprocess regressions and a passing control.

The most important lesson is that integrity and authenticity are different. Hashes do not authenticate report producers. Anyone able to rewrite the manifest and reports can forge acceptance. Trusted CI configuration and artifact collection remain external requirements; passing these tests does not establish production safety.

Next steps and current status

The working prototype and real Duo repair are verified. The MIT-licensed GitLab repository, merge request and CI history are public. The public demonstration shows actual GitLab pages and a real CI rerun in a 1:45 edited walkthrough with English captions: https://www.youtube.com/watch?v=iegtVRBtJmk. Submitted to Life After Code on 16 October 2026. We plan to validate review effort and usefulness with maintainers before claiming time savings or willingness to pay. No customer revenue, productivity gain or contest award is claimed.

Built With

• gitlab
• python