Urumi

• 

  Pipeline and security scans

• 

  Human approval gate

• 

  Canary incident

• 

  Duo self-healing fix

• 

  Guardrail learned from the incident

Inspiration A failed release can wake up a small team in the middle of the night. We wanted to see how much of the work after code is written—testing, security checks, release decisions and incident response—could be handled by GitLab CI and GitLab Duo, without removing people from the decisions that matter.

What it does Urumi runs changes through tests and security scans, packages the application, then checks the new version in a canary against the stable release. It compares error rate, latency and tool success to decide whether to promote, roll back or hold the release.

When a canary fails, CI opens an incident and assigns it to a GitLab Duo flow. Duo investigates, creates a regression issue and proposes a small fix in a merge request. The fix gets its own canary. A person reviews and merges it.

Urumi also demonstrates guardrails that learn: after an incident, it can propose that future changes to the files touched by the fix require human approval.

How we built it We built the pipeline and release checks in GitLab CI, using an ephemeral k3d Kubernetes cluster to test canary releases. GitLab Duo Agent Platform flows handle risk review, incident triage and the postmortem. The canary verdict is based on measured error rate, p95 latency and tool success. Fixes and learned guardrails go through merge requests; they are not applied directly to main.

Challenges we ran into A useful rollback needed more than a failing build: we had to create a realistic regression, send real tool-call traffic to stable and canary versions, and make the verdict depend on their results. We also had to connect CI jobs to Duo flows while keeping credentials out of logs and retaining human review for risky changes.

For the guardrail demonstration, the CI token could not create commits. We opened and merged the guardrail proposal from a workstation using the same code, then verified the before-and-after risk results in GitLab. The token permission still needs to be fixed for fully automatic back-fills.

Accomplishments that we're proud of We ran a complete release incident loop on GitLab: a real canary regression was rolled back, Duo investigated it and proposed a fix, the fix passed its own canary, and a person merged it. We also showed a file changing from low risk to high risk after a guardrail was added based on the incident.

What we learned Automation can do much of the repetitive release and incident work, but it needs evidence to make good decisions. Comparing stable and canary traffic gave the release verdict a measurable basis. We also learned that automation which changes its own security rules should propose a reviewed change, not silently edit the default branch.

What's next for Urumi We would give the CI token the minimum write permissions needed to open guardrail merge requests automatically, then run more incidents to test how well the learned rules generalize. We would also add stricter protected-environment approvals and test longer canary windows with production-like traffic.

Built With

• ai-agents
• ci/cd
• devsecops
• docker
• gitlab
• gitlab-ci/cd
• gitlab-duo
• helm
• k3d
• kubernetes
• model-context-protocol-(mcp)
• python
• typescript
• woocommerce