Release Agent

Release Agent verifies GitLab Duo repairs with frozen behavioral probes and independent container checks, giving teams traceable evidence before human approval.

Comment

Inspiration

AI can produce a convincing repair while missing the behavior that matters: keeping private data private, enforcing permissions, or preventing duplicate writes.

Release Agent grew from the need to make those repairs independently inspectable. The goal was to connect GitLab Duo’s investigation and repair capabilities to observable evidence that a human can review before approving a change.

What it does

Release Agent investigates API regressions, evaluates proposed repairs, and presents the evidence in a reviewer console.

The demonstrated workflow is Path A — Start Fresh, Assisted autonomy:

  1. A human requests investigation through a custom GitLab Duo flow.
  2. Duo publishes bounded, declarative behavioral probes.
  3. Independent execution confirms that the baseline passes and the defective candidate fails.
  4. After human confirmation, Duo proposes a focused application repair.
  5. The verifier evaluates the repair using frozen probes, separate trusted checks, and isolated containers.
  6. The exact repair image is staged in Docker, making its evidence available for human review.

The demonstration uses an intentionally seeded privacy regression in a team task API. An outsider can read a private task. Duo’s repair restores the missing authorization check.

The recorded results are:

  • Baseline: 12/12 behavioral probes pass.
  • Defective candidate: 11/12 pass, exposing the privacy regression.
  • Repair: 12/12 pass.
  • Separate independent checks: 5/5 pass.
  • Exact-image Docker staging: 6/6 pass.

The video also demonstrates a real approval and persisted receipt in a disposable local copy of the archived verification ledger. That action records local review; it does not merge a GitLab MR or deploy to the cloud.

How I built it

The application and verification services use Python, FastAPI, SQLAlchemy, and Pydantic, with SQLite for local evaluation and PostgreSQL exercised in CI.

GitLab Duo runs a custom investigation and repair flow activated through a Mention trigger. Its probes use a constrained declarative format. The verifier freezes their digest before evaluating the repair, preserving the acceptance criteria throughout the comparison.

Docker provides separate application instances for the baseline, candidate, repair, independent checks, and staging. Reviewed build instructions and dependency locks keep candidate-controlled build changes outside the verification path.

GitLab CI/CD runs the test suites, packaged HTTP checks, Docker checks, preservation checks, claim validation, and Terraform validation. Restricted scanner execution uses digest-pinned Semgrep and Trivy for secrets, static analysis, dependencies, and container images.

Development followed a tests-first approach for behavioral changes and security boundaries. Preservation checks protected the existing interfaces, tests, and historical evidence.

DevSecOps lifecycle coverage

Each stage has a documented implementation and evidence scope:

  • Plan: Structured release-request acceptance criteria and recorded Duo investigation of the demonstration MR.
  • Create: Actual Duo-generated probes and a focused authorization repair, retained as public commits.
  • Verify: Frozen probes, independent behavioral checks, and SQLite/PostgreSQL/T1 CI execution.
  • Package: Reviewed Docker builds, immutable image digests, exact-image staging, and packaged HTTP checks.
  • Secure: Actual restricted scanner runtime smoke across four scanner kinds, with secret, SAST, and dependency positive controls.
  • Release: Recorded Docker staging and local human approval. Hosted cloud promotion is tested with simulated providers.
  • Configure: Terraform validated in CI and a Cloud Run service configuration. Live infrastructure is not provisioned.
  • Monitor: Health-monitor and post-deployment check implementations. Scheduled live monitoring requires operator setup.
  • Govern: Revision/image-bound decisions, attestation contracts, preservation, claim checks, and dependency notices. Native approval and pipeline enforcement remain operator prerequisites.

Challenges I ran into

Keeping verification independent. Agent-generated probes can share the repair’s assumptions. The workflow therefore combines frozen probes with a separate trusted suite and fresh application state.

Distinguishing failures from inconclusive execution. A timeout, infrastructure error, or missing result must block progress rather than count as evidence of a regression.

Handling uncertain side effects. The supporting hosted driver needs to recover when a process stops around an external write. Recovery tests include actual process termination against durable simulated providers.

Keeping claims accurate. Recorded GitLab execution, local reproduction, configuration artifacts, and simulated-provider tests provide different kinds of evidence. The documentation identifies those boundaries explicitly.

Accomplishments I’m proud of

The verified main pipeline passed all twelve jobs, including 495 distinct cases executed across three configurations—1,485 successful test executions.

The project preserves public Duo-generated work, independent container results, actual scanner execution, and a reproducible local evaluation path. Credential-free demonstrations also cover known privacy, permission, and retry defects.

What I learned

Reliable agent workflows need explicit acceptance criteria, independent observations, bounded permissions, and recovery rules. A successful tool response alone does not establish that the intended behavior occurred.

I also learned to treat evidence provenance as part of the product. Reviewers need to know which revision, image, environment, and execution produced each result.

What’s next

The next milestone is live acceptance of the supporting hosted Supervised driver: automatic Duo delivery, Cloud Run deployment, post-deployment verification, and recovery under real provider conditions.

Those extensions are implemented and tested with simulated providers. The submitted demonstration remains Assisted, with no production deployment or Google Cloud bonus claimed.

Source code and evaluation

Built With

Share this project:

Updates

Submission history