Patch Jury — Evidence before merge

Patch Jury helps maintainers distinguish a root-cause fix from a patch that only passes the obvious smoke test. Choose one of three original regression fixtures—page counts, decimal rounding, or interval overlap—and compare a buggy base with two competing fixes. A juror workflow proposes boundary inputs, a trusted specification oracle computes expected answers, and isolated Docker containers execute each version. The result is an inspectable verdict with counterexamples, per-group pass counts, source hashes, and downloadable JSON evidence. A unique patch passing the complete evaluated suite is supported; ambiguous evidence stays inconclusive, with the merge decision left to a human.

The live adapter uses NVIDIA Nemotron through Nebius Token Factory's OpenAI-compatible inference endpoint to generate test inputs, while validation, execution, and verdict logic remain deterministic. The model never supplies executable code or expected answers and does not see the competing patches or held-out cases. The current offline demo uses explicitly labeled hand-authored inputs; authenticated Nebius inference is pending the owner's API key and has not yet been demonstrated. Python, TypeScript, and a durable call-budget ledger keep the workflow small and auditable. These synthetic examples demonstrate the review process, not general correctness or benchmark accuracy.

Try it (offline demo)

Install Python 3.12+ and Docker on Linux. Extract the test build, run docker pull "$(cat runner-image.txt)", then python3 -m patchjury --serve. Open http://127.0.0.1:8000, leave Authored examples · offline selected, and click Compare patches. Inspect the verdict, smoke/challenge/held-out counts and counterexamples; export the JSON evidence. Repeat with decimal rounding and interval overlap. No app account is required. Live mode requires a server-side Nebius key and confirmed credits as documented in README. The test build supports bundled fixtures only.

Built With

• agentic-testing
• developer-tools
• docker
• nebius-token-factory
• nvidia-nemotron
• openai-compatible-api
• python
• regression-testing
• typescript