Goodhart: a referee for hands-off GitLab Duo agents

Lets GitLab Duo agents ship to production hands-off by checking every merge request for weakened tests, scans and settings. Blocks the cheat, restarts Duo, and logs every verdict.

Comment

Inspiration

GitLab's tagline for Transcend is "Ship at agent speed. Prove every step." In a hands-off pipeline, the proof is a green pipeline. The agent can edit that proof, though. When a test stands in the way, the shortest path to green is to skip the test, loosen the assert, or tell the scanner to look away.

This is Goodhart's law: when a measure becomes a target, it stops being a good measure. It already shows up in research. In ImpossibleBench (2025), when unit tests contradicted the task, GPT-5 exploited the tests in 54% of tasks and Claude Opus 4.1 in 50%. In the same study, LLM monitors caught only 42 to 65% of the cheating.

We wanted hands-off agents that teams can actually trust, so we built the referee.

What it does

Goodhart sits in front of every merge into main on linkding, an MIT-licensed bookmark manager with over 11,000 GitHub stars. On every merge request it asks one question: did any check that judges the work get weaker?

  • Nothing got weaker. Goodhart approves and merges, and the pipeline deploys to Google Cloud Run (staging, smoke tests, production).
  • A check got weaker and the linked issue did not ask for it. Goodhart blocks the MR and posts a note naming the rule, the file and the line. It then restarts the GitLab Duo Developer flow with one instruction: restore the check, fix the code.
  • The issue really asks for the change. A custom Duo judge flow must quote the exact sentence from the issue. Goodhart then checks that quote, in code, against a notarized snapshot of the issue taken when work started. Only then does the change pass.
  • The change touches the referee itself (its config, .gitlab-ci.yml, deploy/, .gitlab/duo/, AGENTS.md). Goodhart holds it for a person, whatever the issue says.
  • Someone merges around the gate, even the owner. An audit job opens an issue and stops deploys until someone checks it.

Every decision is written as a ledger note on the MR and published on a public ledger page.

One test, three verdicts (all live, all linked)

The same test, test_archive_should_archive_bookmark, went through Goodhart three times:

  1. MR !2. A skip marker was added, the pipeline was green, and no issue was linked. Goodhart BLOCKED it on rule VER-003 with the file and line.
  2. MR !6 (PLANTED). Goodhart blocked it at 10:16 UTC and restarted the Duo Developer flow. Duo committed the fix (9d17b7db, "remove unmandated skip") at 10:17. Goodhart passed and merged it at 10:25 and it shipped to production, with no human action in between.
  3. MR !4. Issue #2 says "Skip test_archive_should_archive_bookmark while the archive view is being reworked." The judge flow answered MANDATED with that exact sentence, the quote matched the notarized issue (sha256 4092cf27), and the MR PASSED.

There was also one real bug, fixed by Duo and shipped. Issue #5: the Tags page said "1 tags total". The issue was assigned to the Duo Developer flow, which opened MR !11 with a fix and new tests. Replay showed 1379 of 1379 existing tests still pass, and 1 new test fails on main and passes on the MR. Goodhart merged it, and pipeline #2973312014 deployed it to staging and production on Cloud Run. The live app now says "1 tag total".

How we built it

Detectors. Python checks over the diff of each MR against main. Five of the nine stages are live today, each with rule IDs:

  • Verify: deleted or skipped tests, loosened asserts, conftest skip hooks, coverage fail_under.
  • Create: special-cased test literals, constant __eq__, sys.exit in app code.
  • Package: unpinned or downgraded base images, loosened pins.
  • Secure: nosec and other suppressions, scanner exclusions, verify=False.
  • Govern: protected paths, AGENTS.md rules, Goodhart's own code.

Replay. This is the proof. Main's whole test tree runs against the MR's code, and the MR's new tests run against main's code. An edited expected value shows up as a proven behavior change. A fix with no failing test shows up too.

The gate runs on protected main. It runs in main pipelines, on a 10-minute schedule and on every push, so an agent's MR cannot edit the referee that judges it. It holds the only token, approves and merges by exact commit SHA, and is the required approver in GitLab's approval rules.

GitLab Duo Agent Platform:

  • The Developer flow. Issues assigned to Duo, plus restarts started by the gate through the Flows API with merge_request_id.
  • A custom judge flow in the AI Catalog. It sees only the notarized issue text and the list of what got weaker, never the MR description or comments the agent wrote. It answers in a JSON block with a nonce.

Delivery. GitLab CI builds the image, pushes it to Artifact Registry and deploys by digest to Cloud Run through Workload Identity Federation. Cloud credentials exist only on protected main. Resource groups and a head check stop an older pipeline from deploying over a newer one.

Evidence:

  • Drills. 11 of 11 planted weakenings caught across the five live stages, every one labeled PLANTED.
  • Backtest. A run over 100 real upstream linkding commits flagged one: a real base image downgrade.
  • Ledger. A static page that publishes every gate run.

Challenges

  • Keeping the referee out of the agent's reach. An MR pipeline runs the MR's own CI file, so the agent could delete its referee. We moved every decision into pipelines on protected main and made changes to the referee human-only (rule GOV-005).
  • Not trusting the LLM. A judge that reads agent-written text can be talked round. Our judge never sees that text, and its answer counts only if its quote matches the notarized issue word for word.
  • Real pipeline races. In the issue #5 run, an older pipeline deployed over the fix and we re-ran the deploy jobs by hand once. We fixed it with GitLab resource groups (newest first) and a head check before every deploy (MR !15).
  • Our own shortcuts got caught. We merged two MRs before Goodhart had ruled. The audit opened bypass issues #3 and #6 and stopped deploys until we checked them.

What we learned

A green pipeline is only worth as much as the checks behind it. Goodhart is mostly deterministic code plus one narrow LLM call, and that turned out to be the right split. Code finds and proves what changed, and the model only answers "did the issue ask for this?", with an answer we can verify.

What was added during the submission period (Path B)

Original project: https://github.com/sissbruecker/linkding. Upstream runs unit and end-to-end tests on GitHub Actions and has no deployment pipeline. Everything in goodhart/, the GitLab CI pipeline, the Duo judge flow, the Cloud Run deployment (deploy/), the drills, the backtest and the ledger were built during the hackathon. linkding's app code was not changed by us. The only app change is MR !11, written by the Duo Developer flow and refereed by Goodhart.

Limits

  • Goodhart defends against reward hacking. It does not defend against a determined attacker running code inside the test process.
  • Four stages are not covered yet: Plan, Release, Configure and Monitor.
  • Changes merged before Goodhart existed are not reviewed.

Credits

ImpossibleBench (arXiv 2510.20270), OpenAI's chain-of-thought monitoring research, Anthropic's research on reward hacking, Skylos on "testwashing", and goalposts. linkding by sissbruecker and contributors.

Built With

  • artifact-registry
  • django
  • docker
  • gitlab
  • gitlab-ci
  • gitlab-duo
  • gitlab-duo-agent-platform
  • gitlab-pages
  • google-cloud-run
  • linkding
  • pytest
  • python
  • workload-identity-federation
Share this project:

Updates

Submission history