Inspiration
A deployment can succeed, automated tests can pass, and a product can still fail its users.
As technical support, I have seen how difficult it can be to connect a customer-reported behavior to a specific code change, reproduce it reliably, and turn that investigation into a regression test that protects future releases.
This inspired Supportability Gate, an AI-assisted release assurance project built around a simple question:
What if release readiness included evidence that the product still behaves correctly for the people who use it?
The goal is to move from reactive troubleshooting toward a repeatable, evidence-driven release workflow.
What it does
Supportability Gate demonstrates how a user-visible application regression can be connected to behavior contracts, automated browser testing, and GitLab release evidence.
The reference application simulates a support case-management interface with multiple user roles and case-detail views.
The demonstration centers on a controlled regression:
- Establish a working baseline in which an authorized Support Agent can access the History tab.
- Introduce a change that hides the History tab from that authorized user.
- Use behavior contracts and automated tests to detect the regression.
- Collect reproducible evidence, including test results and browser artifacts.
- Apply a correction and validate that the expected behavior is restored.
Supportability Gate integrates GitLab Duo Agent Platform through a custom flow called Support Release-Risk Planner.
A human mention in GitLab Issue #3 triggered released flow version 1.0.3. The agent read the incident using get_issue, assessed the customer-behavior risks, and generated a structured JSON proposal selecting two test journeys: Support Agent History and Team Lead status persistence.
A human then reviewed the proposal and approved its admission into deterministic validation. Separately authorized GitLab CI/CD diagnostics detected the History regression and verified the corrected behavior.
The system deliberately uses Assisted autonomy. Duo proposes the regression coverage, while humans retain approval authority and deterministic tooling executes the tests. Duo does not autonomously modify code or initiate CI pipelines.
How we built it
The reference application uses TypeScript, React, Vite, Node.js, and Express.
Vitest provides automated test coverage, while Playwright exercises user-visible browser behavior and produces diagnostic artifacts.
Behavior contracts define expectations that matter beyond whether an API returns successfully or a component renders. The History-tab regression, for example, represents an authorization-sensitive usability failure: an authorized user loses access to an expected workflow.
GitLab provides the development and CI/CD environment, repository history, and evidence trail. The GitLab Duo Agent Platform is the required agentic automation layer.
The implementation is designed to distinguish three things:
- Expected behavior: what the application is required to do.
- Observed behavior: what automated verification actually demonstrates.
- Release evidence: what reviewers can inspect before accepting a change.
This distinction helps prevent an apparently successful pipeline from being mistaken for proof that the software is ready for users.
Challenges we ran into
One challenge was designing a regression that was realistic enough to represent an actual supportability problem but small enough to reproduce consistently during automated evaluation.
Another was coordinating local tests, browser-level verification, GitLab CI/CD execution, and diagnostic artifacts into a coherent evidence trail.
The most important challenge was defining the boundary between AI-assisted analysis and verified automation. A generated recommendation is not the same as a completed action, and a successful local test is not proof that an agent performed the same operation in GitLab.
We treated reproducibility, evidence retention, and human review as fundamental requirements rather than presentation details.
What we learned
This project reinforced that software quality extends beyond code compilation, unit tests, and successful deployment.
Support incidents often contain valuable information about missing behavioral guarantees. Converting that information into explicit testable contracts can help prevent the same defect from reaching users again.
We also learned that agentic DevSecOps workflows need observable outcomes. The value of automation is stronger when a reviewer can inspect what happened, why it happened, and which evidence supports the result.
What's next
Supportability Gate could evolve from a controlled regression demonstration into a reusable release-assurance workflow.
Future improvements could connect actual support incidents to proposed behavior contracts, expand automated coverage across roles and application states, and incorporate additional GitLab DevSecOps checks.
The long-term vision is a release process in which lessons from customer support continuously improve software quality, with agents helping translate those lessons into verifiable safeguards.
Built With
- express.js
- gitlab
- gitlab-ci/cd
- gitlab-duo-agent-platform
- node.js
- react
- typescript
- vite
- vitest
Log in or sign up for Devpost to join the conversation.