Inspiration

COBOL still runs a large share of banking and government batch processing, and the people who can read it are retiring. Translating COBOL to Java is not the hard part for a language model. The hard part is knowing the Java behaves the same, because a silent difference in rounding, truncation or table indexing becomes a wrong number in a real system.

What it does

Parity takes COBOL source, either one program or a whole folder, and asks Nemotron 3 Ultra for Java. It then compiles and runs both programs on the same inputs and compares the output exactly. GnuCOBOL's output is the answer key.

When the outputs differ, an aligned line-by-line diff goes back to the model for another attempt, up to five. The verdict is either "equivalent on the tested inputs" or "not matched". An empty comparison, or Java that does not compile, is never a pass.

  • Folder mode: open a folder, press Convert all, and download the Java as a zip. COPY copybooks are filled in from the folder, and each program runs on its own test inputs.
  • Assistant chat: explains a diff, a run, or a COBOL construct using the run in front of it.
  • Not supported: CALL between programs. Those programs are skipped, and the UI says so.

How we built it

A Python server with a browser UI. The agent loop is: translate, run the COBOL, run the Java, diff, fix. Nemotron 3 Ultra is called through the OpenAI-compatible API on Nebius Token Factory, with thinking mode switched on only for the later attempts. COBOL runs on GnuCOBOL.

Generated Java and submitted COBOL run in child processes with a scrubbed environment, CPU, memory, process and file-size limits, bounded output, and an unprivileged user in Docker. A spend guard and a per-IP daily limit protect the hosted demo. There are 31 unit tests.

Results

  • 40 IBM Project CodeNet COBOL programs: 38 / 40 matched GnuCOBOL on the sample input (about $1.17 at list prices, 642 s).
  • Extra model-generated inputs on the 38 that passed: 137 / 138 matched. One program (p03309) differed on one input, a truncation case.
  • A 300-line nightly account-posting program matches on all 37 output lines in some runs and fails in others. Model output varies between runs, and we report that instead of hiding it.

Challenges

Numeric details caused most failures: implied decimal points (the V in a PICTURE), truncation to the field size, and counters that accumulate instead of reset. Fixes came from reading the diffs, adding explicit COBOL rules to the prompt, and re-running earlier cases to check nothing regressed. The checker also had to be strict: a comparison with nothing to compare, or Java that does not compile, must never show as a pass.

What we learned

Showing the model an exact diff works better than asking it to "try again". Testing the checker matters as much as testing the translator. This is differential testing: it finds mismatches on the inputs that are run and says nothing about the inputs that are not.

What's next

Support for CALL between programs, more COBOL file-handling constructs, larger generated test-input sets, and a stricter sandbox for untrusted use.

Note: the hosted demo runs on free hosting, so the first load can take 30–60 seconds, and it stops calling the model when its budget is used up.

Built With

Share this project:

Updates

Submission history