-
-
Proof Port: the old SAP transaction grades the new app, on tests sealed before the port began.
-
Old and new side by side for customer C100: the old SAP screen on the left, the agent's Fiori app on the right, same tables.
-
How it works: two agents, walled off from each other. The old transaction answers every case.
-
The stage page after a full ZCRED grade: 139 of 140 sealed cases behave the same. The one difference is shown in words.
-
ZCRED's proof page: the score, the seal, every difference, three deliberate breaks, and what wasn't checked.
-
ZMM01, a stranger's MIT code on five screens: 139 of 141. The one leak check match was reviewed by hand.
-
The new ZCRED app: RAP, OData V4 and Fiori Elements on the same tables, built by the port agent.
Inspiration
Large companies run SAP with thousands of custom transactions, the Z code their consultants wrote over twenty years. Moving to S/4HANA and a clean core means rewriting them. With coding agents the rewrite itself has become the easy part. What nobody can sign off is that the new version does what the old one did. Tests written by someone reading the old code share that reader's mistakes, and the people who knew why each rule exists have often moved on.
The old transaction still runs, though, so I made it the judge.
What it does
Proof Port takes a custom SAP transaction (a module pool: screens, flow logic, ABAP, dictionary) and hands back a new implementation of it on the same database tables, with a proof page that says how closely the two behave.
- Extract. It reads the transaction and everything it uses in the customer namespace. SAP's own objects are listed by name and never read.
- Oracle. It replays any sequence of screen inputs through the old transaction by batch input and records what the screen says and every row it leaves in the tables, then puts the tables back.
- Tests. One agent writes test cases from the extract alone, and the oracle answers them. Expected results come from running the old transaction, not from anyone's reading of its code. The cases are sealed: their SHA-256 goes into a public commit before the port starts, so the history shows the tests existed before the code they judge.
- Port. A second agent, walled off from the cases, builds a RAP business object, an OData V4 service and a Fiori Elements app on the same tables. It can ask the old transaction about cases of its own, and run the same grader on them.
- Grade. Every sealed case runs through the old screen and the new service from the same data, and the error messages and every row are compared. Then the port is broken on purpose three ways, to check the cases catch it.
- Proof page. One page per run: the score, every difference in plain words, the seal, the breaks, and what was checked and what wasn't.
Results
Two transactions went through the whole factory on the first night, both on an SAP ABAP developer trial.
ZCRED (mine, change a customer's credit limit): 140 sealed cases, 139 of 140 behave the same. The one difference: for a letter inside a number, the old screen says "Enter a numeric value" and the new app describes the number format. All three breaks were caught, failing 76, 15 and 15 cases that passed before.
ZMM01 (a stranger's code: trymytoolkit/sap-retail-z-sandbox, MIT, by Romain Hecquet; create a retail article with variants on five screens): 141 sealed cases, 139 of 141 behave the same. The two differences: Exit pressed with no article id (the old screen still demands the field, the new app just leaves) and a base unit that doesn't exist (the old screen refuses it, the new app doesn't). All three breaks were caught, failing 44, 33 and 6 cases.
In both runs the leak check found no sign the port agent saw the cases. Each port took its agent about 70 minutes and 45 USD of Claude at list price; each set of cases took 8 to 18 minutes and 3 to 9 USD. Every difference is on the proof pages as it is; none was fixed after grading.
The agents also found things in the old code that a rewrite would quietly change: in ZMM01, Back and Exit on the main screen do nothing (so its "data will be lost" popup is dead code), a list box offers a value the field's own domain rejects, two of its messages can never appear, and two variants with the same id make it crash. In ZCRED, the screen refuses a negative limit before the program's own "cannot be negative" check can run.
How I built it
Claude Code agents do the stages, run headless. Each gets a brief and its own folder, and the port agent's permissions keep it out of the sealed cases, which are also moved out of the project while it works. The factory is Python standard library talking to SAP through the ADT REST interface: it reads sources, runs small generated reports for batch input and table snapshots, and creates the RAP objects for the port agent. The new apps are RAP, OData V4 and Fiori Elements on UI5. A stage page shows the old screen and the new app side by side, runs the sealed cases live and can break the port and put it back.
ZCRED's run is also recorded in 8090's Software Factory through its MCP server: the extract in the knowledge base, requirements with acceptance criteria written from the extract, a blueprint, the port as a work order with the grade as its comment, and the sealed cases as test cases with one graded test run.
Challenges I ran into
- The first port agent found two bugs in my grader. It compared the change log's timestamp by value, and it read tables through a preview that runs out of generated programs after a few cases. Both are fixed; the agent put the data back itself when the second one bit.
- Batch input proves rules, not display. A success message sent just before the transaction ends never reaches it, it stops at the first error, and F4 help doesn't run under it. The proof page says so instead of implying more.
- A leak check that flags everything is as useless as none. The first version flagged message ids inside transcript metadata; the one that shipped looks only at what the agent did and wrote, and the one remaining match was reviewed by hand and recorded.
- Old screens take what fingers type: a leading
!clears a field,10.is not ten,5-is minus five, and this SAP user writes 1.234,56.
What I learned
The old transaction is a better source of truth than its own code. Both agents read the source carefully and still found behaviour only running it shows. And sealing is cheap: a hash in a commit before the port starts turns "trust me" into something anyone can check.
What's next
Run it on one real customer's Z transaction on a sandbox copy of their system, on ECC as well as S/4. Let a case declare starting rows, so change flows like ZMM01's can be tested. Run several transactions in one night.
Built before the event
- The legacy transactions are the input and were made before the event: ZCRED (mine) and ZMM01 (the public MIT repo above). They live on the SAP system; their extracts are in the repo.
- The SAP ABAP developer trial runs on a cloud VM under SAP's licence. It's not in the repo, and neither is any SAP standard code.
- An earlier private prototype of this approach was built 4 to 9 November. None of its code is in this repo; the repo was started at the event and its history shows that. What carried over is what I learned.

