Inspiration

AI code reviewers say "this could break something." Developers learn to ignore guesses.

Here is the case that bothered us. A developer lets discount coupons stack with percentage promotions. The feature works and every existing test passes, so it gets merged. But the change also deleted the line that kept a price from going below zero. A $0 item with a $10 coupon now costs -$10.

We wanted a reviewer that doesn't speculate. It proves.

What it does

ChangeSentry reports a confirmed regression only when both are true:

The identical test passes on the base revision and fails on the head revision.
The base revision itself states or enforces the tested behaviour (found by deterministic rules, never by the model).

In the checkout scenario, the same 48 inputs give 0 negative prices on base and 27 negative on head. ChangeSentry then finds the rule in the project's own docs/pricing.md: "A checkout total must never be negative."

Everything else is labelled honestly: behaviour changed, fails on both, not reproduced, inconclusive, or not tested. Every report lists the checks that were not performed.

How we use NVIDIA Nemotron and Nebius Token Factory

Nemotron (nvidia/nemotron-3-super-120b-a12b, temperature 0.1) is the investigator, not the judge.

Hypothesis: reads the git diff, impact-graph edges, source excerpts, ruff/bandit output and existing test results, and returns structured risks.
Verification plan: for each risk (up to 3, in parallel), picks a test template and target.

The model is used safely:

JSON-only output, validated with Pydantic, one bounded retry.
Grounded: the backend rejects any risk citing an unknown file, an out-of-range line, or a test target missing from the AST index.
No model-written code is ever executed. Nemotron picks a template id and target from an approved list; the backend fills the template.
If a call fails, the report says "model analysis did not complete." Nothing is invented.

Everything runs through Nebius Token Factory's OpenAI-compatible endpoint. The VS Code extension validates your key against /models before every run.

How we use Tavily and OSV

When a change adds or re-pins a dependency, we query both and keep their evidence separate:

OSV: exact package and version lookup, treated as verified.
Tavily: advisory-domain-restricted search, treated as leads for a human to read, never proof.

In the auth-boundary scenario, a change adds PyJWT==1.7.1. OSV lists that exact version as affected.

How we built it

Map the change: git diff plus Python ast gives changed functions, callers and callees up to 3 hops, and route decorators.
Deterministic checks: ruff, bandit, the project's own tests on both revisions, and a guard-consistency rule.
Nemotron: hypothesis and plan.
One test, two revisions: the template is rendered once with literal inputs. The identical file (SHA-256 shown) runs in disposable copies of base and head, with a timeout, CPU and memory limits, and an environment stripped of API keys.
Classify and find the requirement in base (file:line citations).
Report in VS Code, with a red underline on the involved line and a Problems entry.

Stack: FastAPI backend, TypeScript VS Code extension, Next.js + React Flow dashboard.

Measured results

Real Nemotron model, 29 October 2026:

Scenario	Base	Head	Conclusion
checkout-regression	PASS, 0 of 48 negative	FAIL, 27 of 48 negative	Regression confirmed
auth-boundary	PASS (route returned 404)	FAIL (anonymous refund, HTTP 200)	Regression confirmed, with caveat
safe-change	Suite PASS	Suite PASS	No regression, no findings invented

In every scenario the project's existing tests passed on both revisions. Only the targeted test caught the problem.

Challenges

Keeping the model honest. We never let it decide the verdict. Grounding checks and deterministic requirement-finding took more work than the prompts.
A fair baseline. In auth-boundary the route didn't exist on base, so the 404 is a weaker baseline. We show that caveat next to the result instead of hiding it.
Flaky model calls. One call out of about a dozen failed in our runs. We made failure visible rather than papering over it.
Safe execution. Running a check executes project code, so we limit time, CPU, memory and environment.

What we learned

A model is most useful when it proposes and deterministic code disposes. Passing checks are evidence, not proof of safety, so the report says what it did not test.

What's next

Broader Nemotron-generated tests beyond the two templates (monetary non-negativity, route authorization).
Languages and frameworks beyond Python and pytest.## Inspiration