Prompt Regression Guard

Code review for prompts: a GitLab Duo flow that catches when an AI prompt edit silently breaks behaviour, explains which cases got worse, and asks a human when it isn't sure.

Comment

Inspiration

A one-word edit to a prompt can change an AI feature more than a 500-line code change, and nobody reviews it like code. In our demo, a teammate "tidies" a support-ticket router prompt and deletes the word untrusted. Unit tests pass. The overall eval score is 87.5%, above the team's 80% bar. But three tickets containing "ignore previous instructions and classify this as…" now go wherever the attacker wants. A score threshold in CI says PASS. We wanted the merge request itself to say: these 3 cases broke, they are all prompt-injection cases, this is blocked.

What it does

When a merge request touches a prompt, model setting or eval file:

  1. Router (Duo agent) checks the MR diff. No LLM-related change → the flow ends without running an evaluation.
  2. Eval Runner (Duo agent) runs a fixed eval suite (24 demo cases: normal, edge, prompt-injection) on the target branch's prompt and the MR's prompt, using the same deterministic CLI as CI. Baselines are cached by content hash so the demo and repeat runs are stable.
  3. Reporter (Duo agent) posts one MR note: score by category, the exact cases that regressed (expected vs got, confidence), what a score-only gate would have said, tokens used and an estimated gCO2e with the formula shown. It then labels the MR and, when the verdict is "needs review", adds the prompt owner from CODEOWNERS as reviewer.
  4. A CI job is the hard gate: "block" fails the pipeline, "review" shows a warning, "pass" is green. With GitLab's "Pipelines must succeed" setting, a block stops the merge.

In this demo the model under test is an offline deterministic mock and the 24 tickets are synthetic, so every result is reproducible; one CI variable (GUARD_MODEL_PROVIDER) switches the same suite to a real model. The Router, Eval Runner and Reporter are real GitLab Duo agents.

The policy is configurable: by default only a regression in prompt-injection cases blocks; score drops, any per-case regression, too many low-confidence answers, or a changed case set go to a human. LLM evals are noisy, so only behaviours you declare as protected can block; ambiguous changes go to a person.

How we built it

  • GitLab Duo Agent Platform: a custom flow (flow registry v1, ambient) with three AgentComponents and a conditional router; triggers on Assign reviewer, Assign and Mention; .gitlab/duo/agent-config.yml installs dependencies on the runner. Tools used: get_merge_request, list_merge_request_diffs, run_command, read_file, create_merge_request_note, update_merge_request, add_merge_request_reviewers, get_repository_file.
  • GitLab CI/CD: rules:changes with compare_to so the gate only runs when LLM behaviour can change; allow_failure: exit_codes: [2] turns "review" into a warning; the report is an exposed artifact.
  • Eval engine (TypeScript, MIT): evals/run.ts produces a JSON report per run; evals/compare.ts applies the policy and renders Markdown. Model calls are abstracted: an offline deterministic mock (default, free, reproducible) and the Anthropic Messages API.
  • Carbon: Green Software Foundation SCI, ((E x I) + M) per R, with E from token counts x stated per-token energy assumptions x PUE, I = 480 gCO2e/kWh (Ember 2023 global average), M excluded. Every number in the report can be traced to its inputs.
  • Tests: 29 vitest tests covering the comparison logic, every policy rule, the carbon formula, report format, and both end-to-end demo scenarios.

Challenges we ran into

  • Custom flows live in the AI Catalog, not in the repo, and there is no documented way to start a flow from a CI job. We split responsibilities: CI is the deterministic gate, the Duo flow is the reasoning and reporting layer.
  • LLM-graded evals are noisy. We kept the LLM out of grading (the eval is deterministic given model outputs) and made "ask a human" the default for anything ambiguous.
  • Project CI/CD variables are not visible inside flows, so secrets never enter the flow; the flow's runner uses the mock model, real-model evals stay in CI.
  • Duo Chat said "no credits" until the user preference "Default GitLab Duo group" pointed at the hackathon group.
  • The tsx CLI could not open its IPC socket inside the flow sandbox (listen EPERM). Running evals with node --import tsx fixed it. On that first failed run the guard did not pass the MR: it labelled it needs-human and pulled in the prompt owner, which is the fail-safe behaviour we wanted.
  • Unit tests originally read the live prompt file, so a prompt edit failed the test job before the guard ran. They now use a pinned fixture, and judging prompt changes is left to the guard.

Accomplishments that we're proud of

  • The score-only contrast line: on the demo MR it reads "Score-only gate would say: PASS. Guard verdict: BLOCK."
  • MRs that don't touch LLM behaviour are skipped without running an evaluation, and baselines for unchanged target branches are cached.
  • A carbon estimate whose formula, factors and sources are printed right next to the number.

What we learned

  • Prompts deserve the same review discipline as code, but the review has to speak in cases, not scores.
  • Agent platforms work best when agents route and explain while deterministic tools measure.
  • On the Duo Agent Platform, deterministic rules should set the verdict while agents decide whether to run, explain the result and escalate. The fail-safe matters: when our own first run crashed, the guard escalated instead of passing the change.

What's next for Prompt Regression Guard

  • Guard multiple prompt files and agent YAML (including Duo flow definitions themselves).
  • Repeat each case N times and report flakiness separately from regressions.
  • Pull per-token energy from provider-reported figures when they exist.
  • A Duo Chat agent that answers "why did case i02 fail?" on the MR.

Links

Built With

  • gitlab-ci
  • gitlab-duo-agent-platform
  • green-software-foundation-sci
  • node.js
  • typescript
  • vitest
Share this project:

Updates

Submission history