DevOps as a Service

An issue goes in, a reviewed merge request comes out

Comment

DevOps as a Service — an issue goes in, a reviewed merge request comes out

The problem. Writing code got cheap. Everything after the code — triage, planning the change, keeping it inside safe bounds, tests, review, the merge — still eats senior DevOps/SRE hours, and it is exactly the work small teams skip until it breaks in production.

What it does. File an issue. A pipeline of agents takes it from there:

  1. Deterministic guardrail (no LLM). Rejects out-of-scope or unsafe requests before any model is called — including a hard gate when a plan references files that do not exist.
  2. Triage agent — classifies the issue and routes it.
  3. Planner agent — produces a concrete change plan: which files, what tests.
  4. Coder agent — implements the fix inside the plan's bounds; a run cannot pass unless the test count grows past its baseline.
  5. Reviewer agent — independently reviews the diff; the change ships only on approval.

Every run is cost-accounted per issue, retries only transient API failures (429/5xx, max 3, never auth errors), and falls back across models only on model-not-found. All agent roles run the frontier model muse-wonder-1.0 for this entry; a measured per-role split (frontier on coder/reviewer, smaller model on triage/planner) is documented in the repo as the production-scale option.

Proven before GitLab (original MIT project, GitHub). The pipeline has opened real PRs on its own repo (Tiers 1–3) and passed live on three unseen codebases and a 12-family bench of DevOps/SRE failure scenarios: configuration drift, retry storms, connection-pool leaks, disk exhaustion from missing log rotation, false-healthy health checks, inverted alert thresholds, expired CI tokens, upstream API pagination breaks, silently-skipping scheduled jobs, and more — 12/12 families passed live. The imported snapshot's scaffold suite runs 316 tests in GitLab CI, all passing (see below).

What GitLab adds (new in the submission period, Oct 26–27).

  • The project imported to a public GitLab project: https://gitlab.com/spearworks-group/devops-as-a-service
  • GitLab CI as the lifecycle backbone: verify (scaffold suite, 316 tests on GitLab, + demo-repo suite, + a test-count floor gate), secure (GitLab SAST + Secret Detection on top of the pipeline's own deterministic gates), package (wheel + sdist as pipeline artifacts), release (tag-triggered GitLab Release v0.1.0), monitor (eval scorecard regenerated as a CI artifact on every pipeline). First all-green pipeline: https://gitlab.com/spearworks-group/devops-as-a-service/-/pipelines/2992807406
  • The agent loop runs against GitLab merge requests: issue https://gitlab.com/spearworks-group/devops-as-a-service/-/work_items/1 → MR https://gitlab.com/spearworks-group/devops-as-a-service/-/merge_requests/1 — the live run passed with tests 24 → 26 in 20.3 s at ~$0.016 estimated cost, and the MR pipeline is green.
  • GitLab Duo Agent Platform in the loop: the project's Code Review flow reviewed the agent's MR automatically — and earned its keep. It caught a stale "KNOWN BUG" docstring the fix had left behind, and then caught a design flaw in our own CI fixture handling (restoring a seeded test fixture from the moving default branch breaks after merge). Both findings were fixed in-branch; the fixture restore is now pinned to the v0.1.0 tag commit, exactly the decoupling Duo recommended. Duo then re-reviewed and approved the MR. The full exchange is in the MR's notes. Duo runs on the Duo Enterprise seat granted through GitLab Transcend contributor onboarding.
  • Deployed console live for judging at https://grow-syrup-5o7p-8765.sandbox.us-east-1.onlizard.com/?lizard_token=UZAYxxpUIL-BreyC7JuQDnw5TvMSPoH2 (registered as the project's "demo-console" environment) — watch a run move through guardrail → triage → plan → code → review, with per-stage timing and cost. It defaults to mock mode; no credentials are exposed. Hosting: the console runs in a persistent Lizard sandbox (US East) with a 21-day lifetime covering the judging window; verified live from outside at submission time.

Stages covered. Plan (triage/planner agents), Create (coder agent + MR), Verify (CI test suites + floor gate), Secure (SAST, Secret Detection, deterministic guardrails), Package (CI artifacts), Release (GitLab Release v0.1.0; deployed console), Configure (model routing as CI/CD variables; demo-console environment; pinned-fixture pipeline config), Monitor (scorecard artifact + console), Govern (protected main with no direct pushes, ≥1-approval rule — which demonstrably applies to the owner too).

Autonomy level. Hands-off: an issue is the only human input; the loop runs to a merge-ready, Duo-reviewed MR and a deployed console without a person in the middle. (The same pipeline also supports an approve-and-resume supervised mode, shown in the repo.)

Original project (Path B requirement): https://github.com/josephbolus/devops-as-a-service (MIT)

What changed during the submission period (Path B)

Original project (MIT): https://github.com/josephbolus/devops-as-a-service — a multi-agent pipeline (deterministic guardrail → triage → planner → coder → reviewer) that turns a GitHub issue into a tested pull request, built Oct 22–26, 2026 for a different hackathon. All GitLab-facing work below is new, done Oct 26–27, 2026, inside the Life After Code submission period.

New in this period

  1. Import to GitLab — Oct 26, 2026, ~12:03 PM MDT — public project at https://gitlab.com/spearworks-group/devops-as-a-service (57 commits at import; MIT license visible in About). The group namespace (spearworks-group) was flipped to Public for this; the pre-existing spearworks-project in it remains Private.
  2. GitLab CI pipeline (.gitlab-ci.yml, pipeline-as-code) — first green pipeline: https://gitlab.com/spearworks-group/devops-as-a-service/-/pipelines/2992807406 (6/6 jobs). Visible history also includes the honest failures that got it there: three config-invalid pipelines from the first CI draft (a YAML plain-scalar quoting bug), one run where the suite failed exactly 1 of 316 tests because the pipeline's own DAAS_MODEL_* variables leaked into a test that pins env behavior (fixed by making the test job hermetic), and the MR fixture saga in item 3.
    • Verify: scaffold suite — 316 tests as run on GitLab — plus the demo-repo suite (24 tests), on every push and MR, with a test-count floor gate at the 296-test Oct 26 baseline.
    • Secure: GitLab SAST (semgrep-sast) + Secret Detection template jobs, green in the pipeline above; the pipeline's own deterministic guardrail + plan gate run inside the suite.
    • Package: wheel + sdist (daas-0.1.0) built and stored as pipeline artifacts (package:build, green in the tag pipeline).
    • Release: tag-triggered GitLab Release — https://gitlab.com/spearworks-group/devops-as-a-service/-/releases/v0.1.0
    • Monitor: mock-mode eval scorecard regenerated as a CI artifact on every main/MR pipeline (scorecard:mock).
    • Fixture isolation (CI design note): the scaffold suite's mock end-to-end tests run the pipeline against the in-tree demo repo as a seeded fixture. test:scaffold and scorecard:mock restore that fixture from a pinned snapshot (the v0.1.0 tag commit) inside the job; test:demo-repo always tests the branch's real demo repo. See MR !1 for how this design was reached — GitLab Duo's review caught the first version's flaw.
  3. The agent loop on GitLab — seeded issue → live agent run → MR:
  4. GitLab Duo Agent Platform component — the project's enabled Code Review flow reviewed MR !1 automatically: it found a stale "KNOWN BUG" docstring (fixed in commit 28349daf; Duo's re-review confirmed) and a design flaw in the first fixture-restore approach (restoring from the moving default branch breaks after merge) → the restore is now pinned to the v0.1.0 tag commit, per Duo's recommendation. After both findings were addressed, Duo re-reviewed and approved the MR (05:32 UTC, Oct 27). The MR sits open and mergeable — merging it is the one human gate left, by design. Built on the Duo Enterprise seat granted through GitLab Transcend contributor onboarding (approved Oct 26, 2026).
  5. Governance setup — main protected (direct pushes: No one; merges: Maintainers; force push off) and an MR approval rule requiring ≥1 approval. The protection rejected the owner's own API push during the fixture-pin fix, which then rode the MR branch.
  6. Public deployment — the demo console (defaults to mock mode; no credentials exposed) is live at https://grow-syrup-5o7p-8765.sandbox.us-east-1.onlizard.com/?lizard_token=UZAYxxpUIL-BreyC7JuQDnw5TvMSPoH2 and registered as the GitLab environment "demo-console". Smoke-tested from outside (HTTP 200). Hosting: the console runs in a persistent Lizard sandbox (US East) with a 21-day lifetime covering the judging window; verified live from outside at submission time.
  7. Model configuration — all agent roles run muse-wonder-1.0, declared as pipeline-as-code via CI/CD variables (DAAS_MODEL_TRIAGE/PLANNER/CODER/REVIEWER).

Not changed

The pipeline core (daas/ package), guardrail, eval scorecard method, and demo console are the original MIT project, unchanged. GitLab-side additions are exactly: .gitlab-ci.yml, pyproject.toml, run_mr.py (at import), the CI iterations described above, and — on the unmerged MR branch only — the agent's demo-repo fix and its regression tests.

Built With

  • agent
  • api
  • ci/cd
  • duo
  • gitlab
  • lizard
  • meta
  • model
  • muse-wonder-1.0
  • platform
  • python
  • releases
Share this project:

Updates