Lead Hand : rules first, agent second, one human merge

A GitLab Duo flow for legacy SAP code: deterministic checks run first, the agent fixes only what they leave, one person merges, and the release job refuses anything without a passing stamp.

Comment

Lead Hand is a GitLab Duo flow for teams that maintain twenty-year-old SAP customisations (ABAP) and want to move them toward SAP's cloud dialect. The rules run before the agent, the agent changes only what the rules leave, one person merges, and the release job refuses anything without a passing stamp for the exact merged commit.

Glossary: ABAP = SAP's language · ABAP Cloud = the restricted dialect SAP's cloud allows · ATC = SAP's in-system checker · transport = SAP's deployment unit.

What it does

  1. You assign an issue to the flow. "Make this object cloud-ready." That is the whole instruction.
  2. Rules first. The flow runs abap-mcp, an offline open-source ABAP checker: mechanical fixes, a cloud-readiness score before and after, a syntax check, an offline unit run, an identifier-exists check (every name the agent uses must exist in the repository or the bundled released-API snapshot) and a security check. Nothing leaves the runner.
  3. Agent second. The agent edits the smallest thing that clears each remaining finding, re-gates (the tool refuses a fourth round), commits from disk, and opens one merge request. Its description is the release card: verdict first, who asked, who implemented, who decides, cause with file:line, what it hands to a human.
  4. The pipeline stamps. The merge-request pipeline re-runs the gate with the default branch's tooling (the branch cannot bring its own judge), checks card, scope and allowlist, runs SAST and secret detection, and stamps PASS or FAIL for that commit in the Package Registry.
  5. One human merges. Reads the card, merges or declines. A reviewer flow posts an independent reading on @-mention.
  6. The release refuses. On main the release job publishes only if one merged merge request produced the commit, a service account implemented it, a different human merged it, a PASS stamp matches the head commit, its tree and every changed object, and nothing reached main outside a human push or merge. The release includes an offline abapGit bundle and transport note.
  7. The system round trip. checkout.mjs reads a package (or a transport request where the system records them) over ADT into the repository, hashes every part and runs one in-system ATC check as the "before" number; steps 1–6 run unchanged; the manual apply job refuses if the system changed meanwhile, writes back only what the merge request changed, activates, re-runs the same ATC variant and publishes the record beside the release.

Instructions can be ignored by a model. These checks cannot.

A worked example (!6, zcl_nwb_customer_search)

Dynamic SQL built from user input; the security check blocked it. The first attempt did not clear it and was stamped FAIL. I @-mentioned the flow; its second session used a host variable (WHERE name = @iv_name), re-gated to PASS (0 blocking). I merged; desk-release 0.12.6-mr6 followed.

The catch (!9 and !10, zcl_nwb_sku)

The first attempt (!9) cleared all five cloud blockers and passed every check, but it had replaced UNPACK with a string template without WIDTH: the zero-padding was gone, a behaviour change no language check sees. I declined and re-triggered; !10 used WIDTH = 18, passed, and I merged. The gate is a language verdict; the human is there for behaviour.

The refusal (!7, zcl_excel_ole)

A real abap2xlsx class drives Excel through OLE; ABAP Cloud has no released equivalent. 28 cloud blockers, no fix path. The card says FAIL, hands every blocker to a human, and offers the options (keep on-premise, rewrite, retire). I declined. Nothing was released.

The round trip (!11, package ZLV_NWBDEMO)

Four classes checked out of a BTP ABAP trial; in-system ATC before: 2 findings (two failing unit tests). The flow fixed both defects in two rounds, PASS. I merged; the release job published; the apply job wrote the classes back into the same package, activated them, and re-ran the same ATC variant: 2 → 0, both worklist ids on the record. One container, one data point.

Results on the public record (2026-10-24)

Seven legacy objects (five synthetic, two real abap2xlsx classes, Apache-2.0) plus one package of four classes from a live system. Six released (!2, !4, !5, !6, !10, !11). Two refused (!7 above; !8, whose 9 blockers the flow removed before the gate found 56 blocking references to types outside the corpus). One PASS declined (!9). 41 of 122 baseline findings (34%) were rule-fixable, 26 of 62 on the round-trip package alone; from !5 on they are written to disk before any model call. Thirteen gate stamps are on the record. Every direct push to main was refused. The nightly monitor ran.

DevSecOps stages

plan (issue → flow) · create (branch, merge request) · verify (gate, unit run, stamp) · secure (SAST, secret detection) · package (abapGit bundle) · release (stamp-checked job) · deploy (manual apply job, ATC before → after) · configure (CI/CD component, Pages, settings as code) · monitor (nightly drift) · govern (card, decider ≠ implementer, history check).

What already exists, and what this adds

SAP's Custom Code Migration Agent and ADT MCP server apply quick fixes then AI fixes in an IDE against a live system; AWS publishes an open clean-core agent workflow. Lead Hand adds the GitLab lane around the agent: offline gate, invented-name check, a release job that refuses without a human merge and matching stamp, a card recording who asked, implemented and decided, and a measured round trip.

Honest limits

  • The readiness grade is abaplint blocker density, not SAP's Clean Core level; the system's ATC is authoritative. No S/4HANA simplification-item content.
  • The unit runner executes pure logic only. The identifier check blocks unknown SAP class names, not unknown methods on real SAP classes.
  • A PASS is a language-level verdict, not a behaviour proof (!9). Seven objects is a demonstration, not a statistic; the two real classes were the refusals.
  • Only checkout.mjs and the apply job touch an SAP system; the gate, flow and release job never do. The trial records no transport requests, so the transport path is unexercised (docs/BRIDGE.md §6c).
  • Per-merge-request LLM calls and gCO2e are not recorded.
  • Human touches: six merges, three declines, two closed attempts, three re-triggers, one apply job played. The reviewer flow starts only on a human @-mention.

Links

abap2xlsx classes are third-party, Apache-2.0 (provenance in the repo). SAP and ABAP are trademarks of SAP SE; GitLab is a trademark of GitLab Inc.; this project is independent and not affiliated with either.

Technical notes

Why the order of work is fixed

On ABAP, frontier models get roughly one in five tasks right on the first attempt and reach about three quarters only after several rounds of compiler and test feedback (arXiv 2601.15188, 2026). So the checks have to run first and feed back, and the agent has to be stopped from shipping what the checks have not passed. The code is enterprise ERP code that cannot be sent to arbitrary services, so the checks have to be offline.

How it is built

GitLab CI/CD (check → gate → stamp → secure → release → pages), one GitLab Duo custom flow and one reviewer flow created through the GraphQL API (aiCatalogFlowCreate, consumers and triggers), .gitlab/duo/agent-config.yml, a project skill, .gitlab/duo/mcp.json, SAST and secret-detection templates, the generic Package Registry, GitLab Pages, and zero-dependency Node scripts: scripts/desk.mjs (gate), scripts/release.mjs (release gate), scripts/bridge.mjs (abapGit bundle), scripts/pages.mjs (scorecard), scripts/drift.mjs (nightly monitor). Imported as a clean snapshot of abap-mcp v0.12.6 on 2026-10-19; all automation built after Oct 5.

What changed during the submission period (Path B requirement)

Everything outside src/ and docs/ABAP-MCP.md: the flows, the gate and release scripts, the CI configuration and component, the release card template, the corpus, the evidence, the scorecard, the bridge, the drift monitor, the configuration and bridge docs. abap-mcp itself was not changed for the entry beyond one bug fix (CLI output truncated at 64 KiB on pipes). After an independent review on 2026-10-20 the gate gained a syntax check, blocking for unknown SAP class names, allowlist governance and a placeholder check on the card; the release job gained a stamp-bound candidate check and builds everything before uploading anything.

Challenges

The slim Node image has no git and the executor runs git remote set-url after setup. The AI Catalog rejected two documented flow keys (max_cycles, pre_approved_tools), so the round budget moved into the tool. fix --write --json printed instead of writing, so no mechanical fix reached disk until a real run exposed it. Merge-request pipelines ran the branch's own gate script, so a branch could bring its own judge; the gate now checks out its tooling from the default branch. The gate stamped PASS on an object that had not changed, so the 28-blocker class got a PASS once. The "pipeline must succeed" and branch-protection settings are not available to the provisioned role, which is why the release job, not branch protection, is the gate, and why a history check and an out-of-repo monitor exist. The reviewer flow cannot be started by another flow ("insufficient permissions to create a new pipeline"); a human @-mention starts it. The release history check was written with a fixed 100-commit window on a 46-commit branch and refused everything until a review caught it. A broken file scored 100/A because the readiness check skips what it cannot parse; a syntax check now runs first. The flow's first attempt on the seventh object passed every check and had silently changed a padding width; only the human reading the diff caught it.

Accomplishments

One package round-tripped through a live SAP system with the same ATC check before and after (2 → 0); five objects released through a gate that refused every direct push; two honest refusals on real code; one PASS caught by the human and redone; 15 of 60 baseline findings fixed by rule; a release job that will not publish a half-built set; a nightly monitor that ran.

What we learned

The cheapest way to make an agent trustworthy on legacy code is to decide what it is not allowed to decide, and to write the refusal into a job, not a prompt. Every hole in the gate was found by a real run, never by a review. And the one PASS a person caught (!9, the missing WIDTH) is the argument for keeping the merge with a human: the checks prove the code is admissible, not that it is right.

What's next

Exercise the transport path of the round trip on a system that records requests (the trial cannot); run the real class pair against a system ATC variant to calibrate the offline grade (one pair is done: evidence/calibration/); extend the identifier gate with a system object-name index so SAP-namespace names are checked too.

Built With

  • abap
  • abaplint
  • ai-catalog
  • gitlab
  • gitlab-ci
  • gitlab-duo-agent-platform
  • gitlab-pages
  • mcp
  • node.js
  • package-registry
  • typescript
  • vitest
Share this project:

Updates

Submission history