BranchForge

• 

  BranchForge turns one bug into three isolated Nemotron patch experiments and selects the evidence-backed winner.

Inspiration

Most coding agents commit to the first plausible patch they generate. That is fast, but fluency is not proof: a patch can hide precision loss, weaken validation, or pass only the happy path.

BranchForge treats software repair as an experiment. Instead of asking one model for one answer, it makes competing hypotheses visible and forces them to earn trust under the same evidence.

What it does

BranchForge runs a patch tournament:

1. NVIDIA Nemotron creates exactly three distinct, full-file repair strategies with an explicit thesis and risk.
2. One immutable repository state becomes the fair starting point for every branch.
3. All three candidates execute the same regression suite concurrently.
4. BranchForge records test counts, duration, checkpoint, runtime, risk, score, and diff.
5. A second constrained Nemotron pass judges only the observed evidence.
6. A hard evidence floor makes any nonzero test exit ineligible to win.

The result is a reviewable evidence packet—not an opaque model answer.

The demo incident

Ledger Lite has a refund bug at exact half-cent boundaries. Python’s built-in round uses ties-to-even, while product policy requires conventional half-up rounding in the customer’s favor.

The obvious-looking float fixes are dangerous. A hidden regression above 2**53 catches patches that quietly lose a cent through IEEE-754 conversion. Each candidate must also preserve all existing validation behavior. Ten tests per branch make the tradeoffs concrete.

How we built it

• NVIDIA Nemotron 3.5 Lightning on Nebius Token Factory performs two bounded roles: diverse patch strategy and evidence-only judgment.
• The OpenAI-compatible server API uses reasoning-disabled, nonempty structured responses for low latency, followed by strict JSON extraction, unique ID checks, source bounds, and one schema retry.
• The Nebius Sandboxes / ConTree path creates python:3.12-slim once, writes source and tests to a non-disposable checkpoint, then calls base.run(...) concurrently for every candidate.
• A visibly labeled local subprocess path keeps development and the public demo usable while the project’s Sandboxes beta permission is pending. SANDBOX_BACKEND=auto switches to ConTree automatically when access is enabled.
• FastAPI exposes the run lifecycle; a responsive vanilla JavaScript interface renders the execution graph, branch risks, test results, winner, receipt, and verified diff.
• The app ships as a non-root, health-checked Docker image on a public Hugging Face Space.

Safety and reliability

Repository and issue text are untrusted prompt data. Model output is schema-checked and limited to the target source file. BranchForge never executes model-generated shell; trusted code base64-encodes file contents and runs one fixed test command. Secrets remain server-side. The public UI shows whether a run used Nebius Sandboxes or the local fallback, and whether planning and judging came from Nemotron or a deterministic safety fallback.

Challenges

Nemotron 3.5 defaults to extended thinking, which could spend the response budget before returning structured patch JSON. We used its chat-template controls to disable thinking for these bounded stages, hardened extraction against reasoning preambles and fenced source, and added one schema-count retry. That reduced a real end-to-end public run from more than a minute to roughly five seconds.

The second challenge was making branch comparison fair. Every candidate must see identical source, tests, and commands. ConTree’s parent-checkpoint model is a natural primitive for this; the code path is complete, while project permission is being enabled through the Sandboxes beta request.

Accomplishments

• Real Nemotron planning and judging are both visible in the execution receipt.
• Three generated patches run in parallel and produce 30 comparable test executions.
• The hidden large-integer regression rejects a convincing but unsafe float shortcut.
• Seven automated tests cover API health, sample loading, JSON hardening, fenced source handling, branch execution, and evidence-based selection.
• The public repository includes an MIT license, setup instructions, security policy, contribution guide, Docker deployment, and architecture diagram.

What we learned

Alternative generation is only valuable when the alternatives are genuinely different and evaluated under the same conditions. Fast inference matters, but verifiable state matters just as much. Exposing fallbacks and receipts makes an agent more credible, not less.

What’s next

Next, BranchForge will accept arbitrary repositories through a constrained workspace adapter, add property-based and performance tests, record resource metrics per checkpoint, and open a pull request containing the winning diff plus its evidence packet. Once Sandboxes beta access is active, the hosted receipt will show ConTree checkpoint UUIDs and VM isolation directly.

Built With

• contree
• docker
• fastapi
• httpx
• hugging-face-spaces
• javascript
• nebius-sandboxes
• nebius-token-factory
• nvidia-nemotron
• pydantic
• python
• unittest