MR Readiness Reviewer

Inspiration

A green pipeline is useful evidence, but it does not answer every review question. We wanted a small workflow that gathers the available facts, exposes missing evidence, and leaves merge and release decisions with a person.

What it does

MR Readiness Reviewer runs inside GitLab Duo. After a reviewer assignment, it reads the merge request, retrieves its diff, and reads the merge request again before reporting. It checks returned project/MR identity and the full source SHA, compares the MR-associated pipeline SHA, and flags established current blockers as NOT READY. Otherwise it returns NEEDS HUMAN REVIEW. It has no READY outcome because its two tools cannot establish complete job, artifact, retry, or scan provenance.

How we built it

The implementation uses a native Flow Registry v1 YAML definition with two dedicated read tools, fixed project selectors, and alternate URL inputs disabled. MR selection is checked by prompt logic; it is not a hard per-MR capability boundary. The session report links evidence and marks unsupported claims UNKNOWN. Ruby checks validate the flow structure, and ordinary GitLab CI runs validation and Secret Detection. The model has no configured comment, edit, approval, merge, settings, or deployment tool. GitLab itself creates a persistent startup comment when its reviewer integration launches the flow.

Challenges

Initial read routes failed in genuine sessions. A successful canonical-URL diagnostic and a corrected encoded project selector established the supported MR read sequence. The first complete reviewer report then misstated a diff count and exceeded its requested length. We preserved that failure, removed computed diff totals from the report instructions, and checked the revised output against the actual tool responses.

What we verified

Revised session 9795143 completed three successful correlated reads on source 2f54e6720108b76b71a43931a1473efde039595d. Its 159-word report correctly returned NOT READY for a draft MR despite a matching successful pipeline. Ten sampled claims were supported by evidence or explicitly UNKNOWN. It missed the requested one-sentence limits style, so prompt formatting is not presented as guaranteed. Two incomplete-evidence cases produced NEEDS HUMAN REVIEW, including a matching skipped pipeline. A separate matching failed-pipeline fixture produced NOT READY. Ten untrusted instructions were delivered together in one exposure, with only the expected read calls observed. This is not ten independent security trials. The strict zero-comment target failed because of GitLab’s startup comment. The native UI did not support launching the no-MR input test, and fresh setup time was not measured. Detailed evidence and capture limits are documented in the repository.

What we learned and what is next

Tool success, pipeline success, and complete evidence are different claims. The final reread is an advisory snapshot, not an atomic diff-version guarantee. We would next evaluate richer dedicated evidence tools and more independent cases while preserving human control of the outcome.

Built With

• gitlab
• gitlab-ci/cd
• gitlab-duo-agent-platform
• ruby
• yaml