Proof Before Green

Proof Before Green combines GitLab Duo-assisted repairs, agentic CI investigation, independent behavioral tests, and auditable guardrails to verify code changes before they are trusted.

Comment

Inspiration

AI can generate and modify code faster than ever. But faster code is not necessarily correct code.

An AI-generated optimization might pass ordinary tests while silently breaking important application behavior. In systems handling financial data, tenant isolation, or persistent state, even a small regression can have serious consequences.

That raised a question:

What if AI-assisted code changes had to prove behavioral correctness before being trusted?

That is the idea behind Proof Before Green.

We wanted to build more than another AI coding assistant. We wanted a workflow where AI-generated changes face independent verification, operate within explicit boundaries, and produce auditable evidence of what actually happened.

What it does

Proof Before Green is an evidence-first, supervised workflow for detecting software regressions, generating targeted repairs with GitLab Duo, and verifying their correctness before acceptance.

Instead of trusting a code change because it looks plausible or a CI pipeline is green, the system evaluates behavioral invariants, supervises repair proposals, and records the evidence necessary to review the outcome.

A real demonstration: The $7.50 discrepancy

Our demonstration uses a multi-tenant Django application with PostgreSQL and an in-memory reporting cache.

After an additional $7.50 refund, the database's correct refund total is $22.50, but the cached report still returns $15.00.

The cause is a deliberately introduced cache invalidation defect: invalidate_tenant_cache() contains an ineffective pass statement.

The independent evaluator checks eight behavioral contracts, covering correctness properties such as financial totals, tenant isolation, refund freshness, and execution integrity.

Seven contracts pass. One fails: B03 — Refund Freshness.

That single failure is enough to reject the candidate.

How the supervised repair works

Our original demonstration follows an evidence-driven sequence:

1. Detect the regression

An independent evaluator compares observed application behavior against expected results and identifies the failing B03 contract.

2. Provide diagnostic context

A human operator supplies GitLab Duo Chat with the failing contract, relevant context, and the suspected cache invalidation defect.

3. Generate a targeted repair

GitLab Duo generates a structured Python repair proposal that replaces the ineffective invalidation logic with tenant-specific cache eviction.

The fix removes cached reports for the affected tenant without unnecessarily clearing other tenants' cache entries.

4. Enforce safety boundaries

A separate supervisory controller validates the proposal using original file hashes, modification allowlists, change limits, and a strict one-attempt repair budget.

The controller records the attempt and its outcome in an auditable SQLite ledger.

5. Verify the result

The original repaired candidate passes 8 out of 8 behavioral contracts in recorded local evaluation, including B03.

The generated repair remains visible in a public GitLab merge request, which is deliberately left open and unmerged.

This is a human-assisted repair with automated guardrails and independent verification—not an uncontrolled autonomous code deployment.

How we built it

Proof Before Green combines application-level testing, supervised AI assistance, GitLab's developer platform, and an interactive evidence viewer.

Application and verification architecture

The demonstration application uses:

  • Python and Django for the API and application logic.
  • PostgreSQL for persistent, multi-tenant data.
  • An in-memory cache to demonstrate the regression.
  • pytest and an independent behavioral evaluator for verification.
  • Docker for reproducible development and testing.
  • SQLite for the supervisory controller's persistent audit ledger.

The evaluator operates separately from the repair implementation. It uses synthetic fixtures, behavioral contracts, and frozen evaluation assets to help ensure that the repair is measured against established expectations rather than modified acceptance criteria.

Eighteen frozen evaluator assets are verified against recorded integrity information.

GitLab Duo-assisted repair

GitLab Duo Chat generated the original structured repair proposal using the operator-provided diagnostic packet.

Our controller independently checked the proposal's allowed changes, applied the bounded repair in the local demonstration, and preserved the resulting evaluation evidence.

The public GitLab merge request contains the original one-file repair to demo_api/reports/cache.py.

GitLab Duo Agent Platform — Agentic CI investigation

We also used GitLab Duo Agentic Chat to perform a genuine, multi-step, read-only investigation of the project's post-code verification workflow.

The agent autonomously used GitLab's repository, merge request, branch, pipeline, and job-inspection tools to examine source files, CI configuration, pipeline logs, and the original repair.

This investigation uncovered an important gap:

Although the Showcase main branch passed all 115 automated tests, the original repair merge request had never received an independent GitLab CI run.

The agent established that the public CI results applied to main, not to the repaired candidate. It also identified opportunities for MR-specific evaluation, more reliable cache isolation between tests, and stronger merge gating.

This was a separate Agent Platform execution from the original human-assisted repair. It demonstrates agentic post-code investigation through real GitLab tool usage and actionable verification findings.

GitLab CI/CD and public evidence

The public Showcase repository includes the application's source code, CI configuration, evaluation artifacts, and documentation.

GitLab SaaS CI pipeline #2961360893 successfully completed four jobs, including a test suite with 115 passing tests.

These public CI results validate the Showcase main branch. They are separate from the original repaired candidate's recorded local 8/8 evaluation.

Live Evidence Explorer

We deployed an interactive Evidence Explorer on Google Cloud Run.

It allows judges to inspect the recorded evaluation through a visual interface, including:

  • Executive Overview and original failure results.
  • The B01–B08 behavioral invariant matrix.
  • An interactive replay of the recorded evaluation.
  • The GitLab Duo repair diff.
  • The supervisory controller ledger.
  • Raw evidence and downloadable verification artifacts.

The Explorer displays authentic recorded evidence; it is not presented as a live autonomous repair service.

Live application:
https://proof-before-green-explorer-jwlszb5ayq-uc.a.run.app

Public GitLab repository:
https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43377209/showcase

Original repair merge request:
https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43377209/showcase/-/merge_requests/1

Challenges we faced

1. Verifying AI-generated changes objectively

Our biggest challenge was building a demonstration where AI-generated code could be evaluated against independent behavioral expectations rather than subjective judgments about code quality.

We needed to preserve the original failure, isolate the evaluator from the repair, and ensure that proposed changes could not silently weaken the acceptance criteria.

2. Balancing AI assistance with human oversight

We wanted meaningful AI participation without granting unrestricted control over the repository.

The solution was a bounded workflow: GitLab Duo generated the targeted repair, while our controller enforced explicit modification constraints, attempt limits, and evidence recording.

3. Separating different levels of verification

One particularly important challenge was distinguishing local behavioral evaluation from public GitLab CI.

The original repair passed all eight behavioral contracts locally. Separately, the Showcase main branch passed 115 tests in GitLab SaaS CI.

Our later Agentic Chat investigation confirmed that the repair itself had not been validated through an MR-specific CI pipeline.

We chose to document this limitation rather than present unrelated green results as proof of repair-branch verification.

4. Making verification understandable

A technical audit trail is valuable only if others can inspect and understand it.

Building the Cloud Run Evidence Explorer helped transform detailed logs, contract results, code diffs, and ledger records into a coherent demonstration that judges can explore independently.

Accomplishments

We built and published a complete demonstration of supervised AI-assisted repair and evidence-driven verification.

Our verified achievements include:

  • 7/8 behavioral contracts on the original failing candidate, with B03 correctly detecting the stale-cache regression.
  • 8/8 behavioral contracts passing in the recorded local evaluation after the original GitLab Duo-generated repair.
  • 115/115 automated tests passing across four successful jobs in a separate public GitLab SaaS CI pipeline on Showcase main.
  • 18 frozen evaluator assets verified against recorded integrity information.
  • A one-attempt repair workflow with modification restrictions, file-hash validation, and an auditable controller ledger.
  • A public GitLab merge request preserving the original repair.
  • A genuine GitLab Duo Agentic Chat investigation that independently identified a meaningful CI verification gap.
  • A publicly accessible Evidence Explorer deployed on Google Cloud Run.

These results establish behavioral verification, supervised execution, and auditability. We do not claim measured performance, energy, or carbon reductions.

What we learned

A green pipeline is not automatically proof that the specific change being reviewed is correct.

Verification must be tied to the right code revision, the right behavioral expectations, and the right evidence.

We also learned that AI is most valuable when its outputs can be independently challenged.

GitLab Duo helped generate a targeted repair and later investigate the broader CI workflow. The evaluator and controller provided the boundaries and evidence necessary to understand what those activities actually accomplished.

The guiding principle is simple:

AI should accelerate software engineering without weakening our ability to verify its results.

What's next

Proof Before Green is currently a bounded demonstration, but the architecture provides a foundation for broader post-code automation.

Our next priorities are:

  1. MR-specific GitLab CI verification: Evaluate the exact candidate commit and require passing behavioral checks before merge.
  2. Deeper GitLab Agent Platform integration: Connect agentic investigation, repair proposals, and verification more directly through supported agents, flows, and triggers.
  3. Expanded behavioral coverage: Add more invariants for concurrency, authorization, tenant isolation, data integrity, and other regression categories.
  4. Human approval workflows: Add explicit review and promotion controls for higher-risk changes.
  5. Measured optimization impact: Incorporate performance and Software Carbon Intensity measurements so future optimizations can be assessed for both correctness and environmental impact.

Our long-term vision is an AI-assisted DevSecOps workflow where every consequential change produces inspectable proof—not just a success message.

Proof Before Green — Verify correctness before claiming efficiency.

Built With

Share this project:

Updates

Submission history