Inspiration AI coding agents can generate convincing code, but teams still need to understand what changed, whether it works, and who authorized its delivery. Legacy modernization makes that challenge especially clear: a small change in rounding or record handling can alter business behavior.
We built Deterministic Factory to connect AI development with explicit checks, independent verification, and evidence tied to the exact code being reviewed.
What it does Deterministic Factory brings together three workflows:
Understand: Extract source facts, map dependencies and functional areas, and generate documentation and business-rule proposals with source citations. Build: Turn a goal into a specification and scoped tickets, assign native coding agents, run configured checks, and independently review the resulting candidate. Modernize: Prepare legacy behavior references, golden test cases, and bounded rewrite tasks, then compare candidate outputs against the declared reference. An operator console exposes task states, evidence, budgets, event history, and controls for intervention. Automatic publication is restricted to explicitly configured fixture repositories; other repositories require review.
How we built it We used Go for the controller and pipelines, PostgreSQL for durable lifecycle state, and React with TypeScript for the operator console. Tree-sitter extracts source structure, while a Next.js documentation site presents generated understanding.
Claude and Codex run through their native interfaces in separate writable Git checkouts. Builders and verifiers have separate responsibilities, and acceptance evidence records exact revisions and check configurations.
The pipelines exchange artifacts through explicit contracts. For the COBOL modernization target, a Go reference implements documented IBM Enterprise COBOL 6.4 semantics, with GnuCOBOL available as an optional cross-check.
Challenges we ran into The hardest problem was deciding when the system had enough evidence to advance. A worker finishing does not establish that its code passed, and a timeout does not establish that an external operation stopped.
We also encountered unsupported model claims, invalid citations, incomplete language coverage, and subtle legacy arithmetic behavior. These drove explicit failure states, bounded retry loops, source validation, and conservative handling of uncertain outcomes.
Keeping demonstrations honest mattered throughout: simulated transports, prepared responses, reference tests, and live agent runs establish different things.
Accomplishments that we're proud of We connected repository understanding, scoped planning, native agent execution, and independent verification into one inspectable workflow.
The repository records a live fixture build where a Claude builder produced code and a new test, and a Codex verifier brought the candidate to Ready. It also records bounded native fixture results across twelve languages.
For modernization, we implemented reference comparisons, edge cases, external holdouts, mutation checks, and receipts binding artifacts to their source. The complete live COBOL rewrite remains unfinished; the recorded live run stopped at citation validation and refused submission.
What we learned Deterministic checks give AI work a dependable structure, even when model outputs vary. Useful autonomy depends on clear scopes, durable state, independent review, and visible uncertainty.
We also learned that source citations are evidence rather than proof, separate checkouts are not security sandboxes, and passing a finite test corpus does not establish universal behavioral equivalence.
What's next for Deterministic Factory Our next milestone is a complete live COBOL modernization run: validated understanding, a model-built rewrite, independent acceptance, and verification of the exact published fixture commit.
After that, we plan to strengthen worker isolation, qualify hosted integrations, expand language and repository coverage, and improve opportunity discovery around owner-defined goals. Production delivery will require additional compatibility checks, staged rollout, and tested recovery.## What's next for Deterministic Factory