Witness

A GitLab Duo agent for the bug nobody can reproduce: it finds the evidence, fixes the cause and proves the fix.

Comment

Inspiration

A paid checkout sometimes totals R0. But R0 is also the right answer for a genuinely free order, and the logs don't say which happened. Nobody can make the bug happen on demand. Most of the time goes into proving what happened, not writing the patch. The obvious fix, "never allow a zero total", would break the orders that really are free.

What it does

Witness is a hands-off agent on the GitLab Duo Agent Platform. You file an issue and walk away. A controller watching the project picks it up, and Witness then:

  1. asks GitLab Duo for hypotheses and a small, time-limited runtime probe;
  2. waits for the bug to happen and captures the evidence;
  3. replays the evidence to confirm the bug is real;
  4. asks GitLab Duo for a fix and a regression test;
  5. has GitLab CI check the fix with tests the agent cannot edit, including a check that free orders still cost R0;
  6. deploys the checked fix and tests it again over HTTP;
  7. removes the probe and reports back on the issue.

Bad fixes never reach deployment. A wrong fix is stopped by CI before it ships, not rolled back afterwards.

It's for teams that lose time on intermittent production bugs.

The recorded run (issue 4, 21 Oct 2026)

  • 6 min 41 s from a new issue to a verified, deployed fix.
  • Zero human interventions after the issue was created.
  • GitLab Duo proposed three hypotheses. It supported "stale promotion cache", left "clock boundary" untested and rejected "rounding", noting that the free guide is legitimately zero. It chose a 15-minute probe on three code points.
  • The probe caught a paid order (R129.99) coming back as R0. Replay confirmed it.
  • Duo's fix changed two files. CI passed the captured case, an independent regression and all 66 pricing contract cases.
  • After deployment: paid order R129.99, free order still R0. Probe removed; outcome posted on the issue.

Try it yourself

Judges can run the repo's real broken code and the agent's fix in the browser, with nothing to install: see the bug, run all 66 checks, or write their own fix and see whether it passes. Links: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43513500/lights-out/-/blob/main/docs/TRY-IT.md

Live on Google Cloud

The exact fix CI verified (commit 376b0b5, unchanged files with SHA-256 hashes) is live on Google Cloud Run: https://witness-verified-fix-918290555848.us-central1.run.app

It was deployed after the recorded run, from the same verified code. Deploy code: https://gitlab.com/gitlab-ai-hackathon/transcend-october-2026/43513500/lights-out/-/tree/main/deploy/cloudrun (shared with our second entry, Lights Out, because Witness's main branch is locked to its recorded, verified owner code). The recorded hands-off run deployed to a local service, and no agent run was repeated.

Wrong fixes the checks catch

We ran six plausible human-style fixes through the same checks CI used (the 66 pricing cases plus the captured issue-4 order). Added 27 Oct 2026; these are hand-written test fixes, not agent output.

Fix Pricing checks Issue-4 bug fixed? Result
Never allow a zero total 42/66 no rejected
Reject any zero total as an error 42/66 no rejected
Cap discounts below 100% 57/66 no rejected
Hard-code the launch-pack price 58/66 yes rejected
Shorten the cache to 1 second 64/66 no rejected
Ignore the cache completely 66/66 yes accepted (genuinely correct, just slower)
Original broken code 64/66 no rejected
Agent's fix 66/66 yes accepted

The checks judge behaviour, not code style: every fix that breaks a legitimately free order, or leaves the bug, is stopped before deployment.

How we built it

  • Python (standard library) for the controller, the probe SDK, the checkout fixture and the local service.
  • GitLab Duo Agent Platform for the two proposals: hypotheses and probe, then fix and test.
  • GitLab CI/CD runs the checks the agent can't edit, on a separate owner verification ref.
  • The agent's code runs only in a locked-down, network-disabled Docker container. The agent never gets Git or API credentials.
  • SQLite holds the controller's journal and a hard budget on Duo runs.

Challenges

It didn't work first time, and we kept every attempt on the record. The six budgeted Duo runs covered four issues:

  • issue 1 failed at startup;
  • issue 2's answer was 5 characters over a schema limit and was rejected;
  • issue 3 produced a fix but stopped on a Git lookup before publishing. It was later finished in a separate, supervised recovery with no new Duo runs, and is not counted as hands-off;
  • issue 4 was the fresh hands-off success.

Accomplishments we're proud of

  • An agent that works out which zero is the bug before it changes anything.
  • Checks the agent can't edit, including a legitimately free order, so "make zero impossible" fails.
  • A full loop from issue to deployed, re-checked fix with no human in it, and judges can test the result themselves.

What we learned

The hard part of agent repair isn't writing the patch. It's evidence, and checks the agent can't influence. A small, expiring probe plus an owner-written contract made the agent's fix provable.

Limits (honest)

  • One synthetic checkout app with an injected business clock.
  • The recorded run deployed the fix to a running local service on Chad's Mac after CI, not to production. That service has since stopped. The same verified fix now runs on Google Cloud Run (deployed afterwards, not by the agent run).
  • Nothing was merged: the exact commit CI checked was deployed.
  • The Duo session pages need a GitLab sign-in. Duo's responses are copied publicly into the repo.

What's next

More bug types, deployment to a real environment with rollback, and opening a merge request with the evidence attached.

Links

Built With

Share this project:

Updates

Submission history