Red to Green

An assisted incident-learning prototype turning failed GitLab CI pipelines into human-approved fixes and reusable team takeaways with GitLab Duo.

Comment

Inspiration

In continuous integration and modern software delivery, pipeline failures happen every day. Too often, when a build breaks, developers waste valuable time manually combing through thousands of lines of container logs, guessing the root cause, or applying ad-hoc patches without recording what went wrong.

We were inspired to build Red to Green to turn routine pipeline failures into a repeatable, assisted incident-learning loop. Using the GitLab Duo Agent Platform, our goal was to bridge the gap between automated test detection, AI-assisted diagnosis, strict human review, and persistent team memory.

What it does

Red to Green demonstrates an end-to-end assisted incident response workflow centered on an intentional Python shopping-cart regression:

  1. Automated CI Failure (RED): A commit triggers an automated GitLab CI pipeline running pytest. Item prices of 1,299¢ ($12.99) and 499¢ ($4.99) expect a total sum of 1,798¢ ($17.98). An injected fee surcharge of + 100 cents causes the assertion to fail with 1,898¢ ($18.98).
  2. Manual Duo Investigation: The developer opens native GitLab Duo Chat directly on the failed GitLab Runner job trace and asks the CI Expert to identify the issue.
  3. Root-Cause Analysis: GitLab Duo analyzes the container logs, explains the cents arithmetic failure, and suggests a minimal, one-line revert patch in cart.py.
  4. Person-in-the-Loop Approval: Enforcing strict human oversight, GitLab Duo generates an interactive tool approval card (Allow Gitlab add commit). The developer inspects the diff and authorizes the commit.
  5. Automated Verification (GREEN): GitLab CI automatically verifies the fix in a fresh runner environment, passing all 25 unit and regression tests. The developer confirms the merge request into the protected main branch.
  6. Incident Learning Tracker: A local companion dashboard built with FastAPI and React allows teams to record timeline events, review diffs, and preserve postmortem takeaways for future onboarding.

How we built it

  • GitLab CI/CD: Automated runner pipeline (.gitlab-ci.yml) executing Python 3.11 test suites and exporting JUnit XML test reports.
  • GitLab Duo Agent Platform: Native GitLab Duo Chat (powered by Claude 3.5 Sonnet) leveraging the CI Expert agent and interactive tool authorization cards for safe code modifications.
  • Backend: FastAPI with SQLite and SQLAlchemy (backend/api/incidents.py and backend/api/webhook.py) to structure incident schemas, timelines, and postmortem records.
  • Frontend: React, TypeScript, and Vite styled with Vanilla CSS and Lucide icons, providing an incident timeline and learning metrics view.
  • Testing & Verification: Pytest suite (test_cart.py and test_local.py) containing 25 automated tests verifying math logic, API routes, and synthetic sample isolation.

Challenges we ran into

  • Ensuring Strict Truth-in-Evidence: The biggest challenge was balancing ambitious automation ideas with verifiable reality. Rather than overclaiming that the agent operates completely unassisted or faking automation with background curl scripts, we deliberately framed the project around GitLab's official Assisted autonomy tier, ensuring every claim in the documentation matches the actual recorded workflow.
  • Designing Safe Human Gates: In an era of autonomous coding agents, preventing unvetted code from reaching production is critical. We designed the workflow so the agent can never bypass human approval or push directly to protected default branches.
  • Video Timing: Distilling the entire incident lifecycle—from test failure to Duo diagnosis, code inspection, human approval, and green CI verification—into a crisp, rights-cleared walkthrough under the hackathon's strict 3-minute limit.

Accomplishments that we're proud of

  • Capturing an authentic, un-simulated native GitLab Duo Chat interaction on a real failed GitLab runner job.
  • Maintaining a 100% green pipeline history on GitLab CI with 25 passing automated tests.
  • Producing a clean, 2-minute-and-40-second video demo with 100% original narration, zero copyrighted background audio, and explicit human authorization gates.
  • Creating a modular educational prototype that any developer can clone and run locally with Python and npm.

What we learned

We learned how powerful foundational agents like GitLab Duo's CI Expert are when grounded in real runner logs and compiler traces. More importantly, we learned that the highest-leverage application of AI in CI/CD is not unattended automation, but augmenting human developers: accelerating the path from a red pipeline to a green build while capturing institutional knowledge for the team.

What's next for Red to Green

  • Cloud Deployment: Deploying the FastAPI backend to Google Cloud Run to ingest live, real-time GitLab webhook event streams.
  • GitLab Agent Platform Custom Flows: Activating event-driven Flow Triggers once custom YAML flows become generally available on GitLab SaaS.
  • Team Memory Intelligence: Expanding the tracker database to cluster recurring pipeline failures and suggest historical fixes based on past postmortems.

Built With

Share this project:

Updates

Submission history