Relay Pump

• 

  Catch the risk: Relay Pump reviews bounded release evidence and returns an explainable HOLD when the proof is not strong enough to ship.

• 

  Repair and re-check: Codex repairs the tie-breaker and adds four focused tests; Relay Pump moves the same task from HOLD 32 to SHIP 96.

• 

  Evidence, not a code dump: process and task opt-ins gate an auditable release decision. In this demo, only release-evidence.json was sent.

Inspiration

I’ve been programming for 14 years and now work in medicine. In both, "done" is not the same as evidence.

Coding agents can produce a patch quickly, but test output, review notes, known risks, and rollback instructions often remain scattered across terminals and chat. I built Relay Pump to put that evidence in one place before a person decides to ship.

What it does

Relay creates a task folder for each change and can provision an isolated Git worktree. Codex implements the work, while Relay records the acceptance criteria, patch, test output, review findings, risks, and rollback plan.

GPT-5.6 can then review one deliberately prepared release-evidence.json file and return a schema-validated HOLD or SHIP recommendation. It cannot run commands, modify Git, merge code, or deploy anything. Relay saves its findings and their resolution history, but the release decision stays with a person.

The proof

I tested Relay Pump with a plausible queue-sorting bug. The implementation used Python’s randomized hash as a tie-breaker and had only a basic priority test.

GPT-5.6 returned HOLD 32 and identified two blocking problems at queue_sort.py:9 and test_queue_sort.py:6. Codex replaced the hash with the original source index and added four focused tests covering priority ranks, creation-time ordering, a three-item exact tie, and input immutability.

After the findings were resolved and the rollback was documented, the same task returned SHIP 96. The first attempt did not ship, which was the point: better evidence changed the decision.

How GPT-5.6 is used

GPT-5.6 reviews the implementation evidence, tests, developer findings, risks, and rollback plan together. It returns structured JSON rather than a conversational answer, and the request uses store: false.

Its verdict is advisory. Relay does not treat a model response as proof that a claimed test actually ran.

How Codex was used

The original Relay task queue existed before Build Week. During the event, I used Codex to add the GPT-5.6 integration, response schema, evidence boundary, persistent review comments, resolution workflow, tests, reproducible proof runner, and judge-facing demo.

Privacy and safety

Cloud review is off by default and requires both process-level and task-level opt-in. Only a size-limited evidence file marked safe by its author can be sent. Arbitrary repository files are not uploaded, and sensitive, clinical, PHI-bearing, or secret material must remain local.

What I learned

The difficult part was not getting a model to produce a score. It was deciding what the model could see and what its recommendation could change. Early HOLD results forced me to replace summaries with concrete patch, test, review, risk, and rollback evidence.

What’s next

Next I would add signed test provenance, repository-specific evidence adapters, and consistency testing across repeated reviews.

Built With

• python