NVIDIA Nemotron generated the candidate. Nebius sandboxes executed BASE FAIL, CANDIDATE PASS and COUNTERFACTUAL FAIL.
Public receipt: SHA-256 digest, source commit, VERIFIED result and recorded Nebius execution telemetry.
Real Tavily Search retrieved three CVE sources. Grounding was BLOCKED; candidate execution was NOT_RUN.
Tavily grounds a frozen contract; the authority firewall prevents web claims from overruling code execution.

Inspiration — Green Tests Are Not Proof

AI coding agents can generate patches in seconds. But there is a critical gap between producing a change and proving that the change actually fixed the intended problem.

A green CI pipeline does not establish causation. The tests might have passed before the patch. Assertions may have been weakened. An agent may produce a plausible change that does not address the underlying defect.

Basebreak was built around one principle: If the patch matters, the base must break.

We wanted a verification runtime that does not simply ask whether an AI-written patch passes tests, but independently investigates whether the patch caused the required behavioral transition.

What It Does — Causal Verification Across Three Worlds

Basebreak separates code generation from independent verification.

For a behavioral bug fix, it evaluates the same sealed behavioral witness across three isolated execution states:

BASE — FAIL: The trusted original code must reproduce the defect.
CANDIDATE — PASS: The AI-written patch must satisfy the witness.
COUNTERFACTUAL — FAIL: Removing the candidate patch delta must reproduce the failure.

Only a consistent, independently executed transition can support a VERIFIED causal verdict.

This prevents an AI coding agent from certifying its own output simply because its development tests are green.

How We Built It — Nebius, NVIDIA and Independent Execution

Basebreak's candidate-generation workflow uses NVIDIA Nemotron 3.5 Lightning (nvidia/Nemotron-3_5-Lightning) through Nebius Token Factory.

The trusted Python verifier statically inspects generated code before isolating execution. Nebius Token Factory Sandboxes provide separate execution environments for the base, patched candidate and counterfactual states.

Our recorded ResetVault demonstration — a synthetic security-sensitive token-vault scenario — includes genuine Nebius inference and sandbox execution.

The verified historical run produced:

BASE: FAIL, exit code 1.
CANDIDATE: PASS, exit code 0.
COUNTERFACTUAL: FAIL, exit code 1.
Overall verdict: VERIFIED.
Provenance at execution: LIVE_NEBIUS.

The workflow recorded 11 Nebius sandbox operations and bound the executed source, candidate tree, witness outcomes and execution facts to a cryptographic receipt.

The public evidence layer uses Cloudflare Workers and a SQLite Durable Object ledger. Its read-only Judge dashboard lets anyone inspect historical verification results without an account or paid API access.

Receipt integrity is checked using SHA-256. The verifier also produces a custom RSA-2048 proof; this is not presented as a standard RSA-PSS or PKCS#1 signature scheme.

Meaningful Tavily Integration — External Facts Cannot Overrule Execution

Some engineering contracts depend on external facts, such as security advisories or breaking library changes.

Basebreak integrates Tavily Search to retrieve and bind advisory information to a frozen engineering contract.

A genuine bounded Tavily Search request for CVE-2024-21626 returned three external sources, including GitHub Advisory Database and Snyk.

The deterministic grounding firewall returned INSUFFICIENT_GROUNDING / BLOCKED because the required freshness and trusted-observation conditions were not fully satisfied. Candidate execution for that advisory scenario remained NOT_RUN.

This is an intentional authority boundary, not a fabricated verification success. Web search can inform what needs to be tested; independent execution determines whether code behavior has been verified.

The Tavily advisory scenario is separate from the successfully verified ResetVault demonstration.

Challenges We Overcame

The hardest challenge was establishing trustworthy boundaries between the AI builder, independent verifier, cloud sandboxes and publicly displayed evidence.

We also encountered practical platform challenges:

Nebius Token Factory onboarding required payment-card configuration before promotional credits could be activated.
Sandboxes initially returned HTTP 403 because project-level beta permissions had not yet been enabled.
The initial public Judge presentation displayed safe NOT_RUN placeholders despite authentic historical receipts being available. We repaired the read-only receipt binding and validated the resulting signed-out presentation.
We had to preserve precise distinctions between mock fixtures, local execution, genuine live provider calls and replayed historical evidence.

These constraints influenced Basebreak's fail-closed design.

What We Are Proud Of

Basebreak delivers more than an AI-generated code suggestion.

It demonstrates a real NVIDIA Nemotron-generated candidate evaluated through Nebius execution environments, with independently checked causal outcomes and publicly inspectable evidence.

Our public Judge interface connects an understandable three-world result with its execution provenance, source identity, receipt digest and model telemetry.

The core idea is deliberately simple: a patch should not earn trust merely because a dashboard is green.

Potential Impact

Basebreak targets teams using AI coding agents for bug fixes, security-sensitive maintenance and automated software engineering.

As AI agents become more autonomous, verification needs to become more independent. Causal witnesses and counterfactual execution can help reviewers distinguish genuine behavioral changes from coincidental green tests.

The long-term opportunity is to make evidence-backed verification a reusable component of agentic software engineering pipelines.

What We Learned and What's Next

We learned that credible AI engineering requires strict separation between generation, execution and authority.

Our next priorities are broader real-repository coverage, more adversarial witnesses, improved deployment reproducibility, standardized cryptographic signing and additional evidence-driven evaluation scenarios.

Current boundary: Basebreak demonstrates empirical causation under specific witnesses; it does not claim universal formal correctness. The public Judge displays previously recorded authentic Nebius execution, not unrestricted on-demand provider execution.