Post-Close Auto-Checks

An issue stays closed only as long as production still does what the issue asked for.

Comment

Track and level: Path A (new project). Level entered: Supervised. In both fire drills a person started the flow that reopened the issue; only the closing in drill 2 happened with no one involved.

Inspiration

Closing an issue records that a change was merged. It says nothing about whether the change still holds a month later. The theme "Life After Code" asks what happens once the code is written. Our answer is one rule: an issue stays closed only while production keeps answering what the issue asked for.

What it does

A person writes an issue from the intent template: a sentence of intent and a promise block, for example GET /status.json, status 200, json-has version. Each delivered issue leaves that promise in promises/, and the live site is checked against it every day.

  • Plan, Create. A GitLab Duo flow (the foundational Developer flow, guided by AGENTS.md) turns the issue into a merge request that adds the promise file and changes the site. Issue #3, an About page, was delivered this way.
  • Verify, Secure. Unit tests run, and every promise is replayed on the built site. GitLab's SAST and secret detection templates also run; they are allow_failure, so they inform and do not block.
  • Govern. The rule (promises are append-only, agents stay out of the pipeline and tooling, the promise equals the issue's block) is checked on the branch by govern:policy, judged again from the default branch by govern:verdict, and audited after each push by govern:audit. The verdict says MERGE-OK, WAIT or REFUSED; it merges nothing. A person, or their agent, merges.
  • Package, Release, Configure. Each push on main uploads the built site as a versioned archive to the generic package registry, cuts a tag and a release, and publishes to GitLab Pages in a production environment.
  • Monitor. A pipeline schedule runs every day at 09:43 UTC. Its monitor:watch job replays every promise on the live site and reads the state of each issue without a token (the project is public). When production and the tracker disagree, the job fails and leaves the comment that reopens or closes the issue in its log and in watch-report.md. The job writes nothing itself.
  • React. A custom GitLab Duo flow, "Reopen broken promise" (id 1018262, version 1.1.0), has the trigger "Pipeline events: Failed". When the watch fails, the flow reads the report and posts the comment on the issue: it reopens #2 with the evidence, or closes it once the promise holds again.

What ran on GitLab: delivery pipelines on main since 2026-10-12, with build, tests, promise checks, audit, package, release and Pages; the daily schedule; the custom flow and its trigger; two fire drills, recorded step by step in DRILLS.md.

Fire drill 2 (2026-10-15). A commit marked [drill] published the site without status.json, which broke the promise of issue #2 (the file answered 404). The watch went red (pipeline 2949533349). The trigger started the flow, but that session ended at start with no output. The flow started by hand posted the evidence and reopened #2 (session 9511530). Commit b8f06f1b republished the full site. The next watch failed with "kept again, close #2" (pipeline 2949592249), and the flow, started by its trigger alone, closed #2 (session 9511775).

How we built it

The project was built and written with AI agents under the entrant's supervision: Claude (Anthropic) wrote the application, the tests, the pipeline, the agent instructions, the documentation, this text and the video script; GitLab Duo Agent Platform delivered issue #3 and runs the reopen and close flow. The first version was written offline from the platform documentation, then set up on GitLab following SETUP.md.

  • sitegen/build.py: the application, a static site generator (Python standard library only) that also renders the public ledger at /promises/ and status.json.
  • tools/ledger.py: five commands, check, policy, watch, verdict and audit.
  • .gitlab-ci.yml: the whole chain, one job per lifecycle stage, named by its stage.
  • AGENTS.md and .gitlab/duo/flows/: the agents' instructions and the flow sources.

The entrant has the Developer role in the project the hackathon provides. That role cannot create a project access token, a CI/CD variable or a branch protection rule, so the design uses none of them. Jobs authenticate with CI_JOB_TOKEN only, where GitLab documents it.

Challenges we ran into

  • No token. The job token cannot touch issues, so the watch only reports and a Duo flow posts the comment. This split is why a "pipeline failed" trigger is the link between the two.
  • The gate lived in the branch it guarded. A branch could rewrite the pipeline. The verdict now runs on the default branch, from its own copy of the rule, and an audit after each push catches a change to an older promise.
  • An outage would have reopened every issue. A site that is down, slow or answering a server error gives "unknown": the job fails and asks for nothing.
  • A repair that broke things again. In drill 1 the [drill] marker counts anywhere in a commit message, and the first repair quoted it, so it republished the broken site. Duo's own repair (!2) was refused by govern:policy because it edited a protected file, which the operator had wrongly asked for.
  • Duo sessions that do nothing. Three of the six Duo sessions on 2026-10-15 ended at start with no output, whether started by hand or by the trigger. We do not know why. The chain still holds because a failed watch stays red until someone acts, but today the reopening is not reliable without a person watching.

Accomplishments that we're proud of

  • A full round trip on GitLab: a broken promise detected, its issue reopened with the evidence, a repair, and the issue closed again by a flow that no one started.
  • A rule an agent cannot quietly weaken: promises are append-only, and the policy refused an agent's merge request that touched a protected file.
  • Documentation that records what went wrong, including the operator's two mistakes in drill 1 and the empty Duo sessions.

What we learned

  • A watch that writes nothing is easier to trust. It fails, explains, and leaves the action to a flow or a person.
  • Failing on "close again" matters as much as failing on "reopen". The trigger only sees failed pipelines, so a restored promise has to turn the watch red once for the issue to close.
  • A Duo session that starts is not a Duo session that works. Every claim about the flow is checked against the issue's history.

What's next

  • Find out why half the Duo sessions end at start, then decide whether the flow can be trusted alone.
  • Confirm that the trigger fires on a scheduled watch (seen so far on a watch started by the API and on a push pipeline).
  • Try intent-to-merge-request.yaml, the custom delivery flow, which is not yet validated.
  • Back the rule with GitLab settings (branch protection, code owner approval) on a project where the role allows it; here the audit detects but does not prevent.
  • Promises are an HTTP status and text or JSON keys. Richer checks would prove more about behaviour.

AI disclosure

This project was built and written with AI agents, Claude (Anthropic) and GitLab Duo, under the entrant's supervision. The entrant chose the idea, created the accounts, approved what was kept and started the steps that needed a person. Claude wrote the code, the tests, the pipeline, the agent instructions, the README and the other documentation, this description and the video script. GitLab Duo wrote the merge requests for issue #3 and !2, and reopened and closed issue #2. The demo video, also assembled by Claude from screenshots of the real GitLab pages, has burned-in English subtitles and no voice.

Built With

  • claude
  • gitlab
  • gitlab-ci
  • gitlab-duo
  • gitlab-pages
  • python
Share this project:

Updates

Submission history