ReleaseProof

Inspiration

A producer can pass its own tests while breaking a legacy consumer. A formatted dollar total looks correct, but deleting the integer total_cents field breaks downstream clients. Release engineers need evidence of both contracts and an explanation they can review before approving a release.

What it does

ReleaseProof is a small, reproducible compatibility lab. It runs actual producer and consumer subprocess suites against fixed synthetic fixtures, generates commit-bound evidence, and independently verifies artifact bytes. The broken fixture passes the producer suite and fails four consumer cases, producing BLOCKED. A deterministic reference repair passes both suites but remains AWAITING_GITLAB_HUMAN_REVIEW. Local caller flags or reviewer names cannot grant approval.

A GitLab Mention triggers a custom read-only Duo flow. Its real completed session read releaseproof.py, diagnosed the missing legacy field, proposed a minimal patch as text and listed regression tests and human validation. The model explicitly distinguished a proposed patch from a performed repair. It correctly explained that a green lab pipeline proves detection of the broken fixture, not compatibility of that fixture.

How we built it

Original Python standard-library code and synthetic data were created during the contest period. Four money cases exercise new and legacy contracts. Subprocesses have a five-second timeout and minimal environment. The manifest binds sources, results and harness hash to the full Git SHA. GitLab CI runs 14 tests, generates the evidence and verifies it. The custom AgentComponent exposes only read_file and list_dir. Mention is its only active trigger. Its service account uses GitLab composite identity and was added only to Showcase.

Challenges and lessons

AI explanations, measured tests and release authority are different kinds of evidence. ReleaseProof keeps them explicit. The successful model session ran no tests and wrote no code; the CI results came from actual execution. The reference repaired fixture was deterministic, not generated by Duo. Artifact verification assumes a trusted reviewed harness: it is not a signed attestation or a sandbox for malicious Python. No production service integration, deployment or release-approval API is claimed.

Accomplishments

14 meaningful tests cover compatibility, money and privacy regressions, crashes, retry bounds, forged approval, stale revisions and tampering. The final integration head passed CI in 16 seconds. The real Duo runner completed in 64 seconds with one read action. All original source, configuration and run instructions are public under MIT.

What is next

Integrate real producer and consumer repositories, compare the harness against an external trusted digest, and require authenticated review of the exact candidate SHA. These are future work, not implemented features.

Try it

Python 3.12 and Git: run python releaseproof.py test, demo, verify. Public MIT repository and CI evidence are linked below. This is a synthetic trusted-harness lab, not a production service.

Built With

• gitlab
• python