Culprit: find the commit that broke it

The bug: some bills are ₹1 short, somewhere in 200 commits.
Step 1: a test that reproduces the bug. It fails with 42, expected 43.
Step 3: the culprit, confirmed on both sides.
Step 2: scout and parallel bisection. 8 commits tested at once per round.
Step 4: why it broke. Python’s round() rounds 42.5 down to 42.
Step 5: minimal fix, robust fix and git revert compared. The revert fails.
Step 6: the verified patch with its regression test.
The shareable HTML report from a real offline run.
GitHub
How it works, and which Nemotron model does what.

Inspiration

"It worked last month." Every developer has heard it, and it usually costs an afternoon: write a test that shows the bug, run git bisect one commit at a time, read the diff, then fix it by hand. We wanted an agent that does the whole loop and checks its own work by running real tests instead of trusting text.

What it does

You give Culprit a git repo and a bug report in plain English. It:

Reproduces the bug. NVIDIA Nemotron 3 Super writes a pytest test. Culprit only accepts it if it fails on the broken commit and passes on an older one. If not, it sends the real pytest output back and asks again.
Scouts history. It tests commits 1, 2, 4, 8… steps back to find a version without the bug, so you don't even need to know a "good" commit.
Bisects in parallel. Instead of one commit per step like git bisect, it tests up to 8 commits per round, each in its own isolated sandbox. Every result is PASS, FAIL or SKIP. If old code fails in a different way (say, an import error), that counts as "can't tell", not "bug present", so Culprit doesn't blame the wrong commit.
Confirms the answer. It re-runs both sides of the boundary twice and warns if the result is flaky.
Explains it. Nemotron 3 Ultra reads the guilty diff and explains the root cause in plain English: the exact lines, why the tests missed it, and how to fix it.
Fixes it, and proves the fix. Nemotron 3 Super writes a minimal and a robust fix, and Culprit also tries a plain git revert. Each candidate runs against the new regression test and the full test suite, compared with a baseline. The smallest fix that passes wins.
Reports. You get a self-contained HTML report (hunt replay, explanation, fix comparison, regression test, model usage and cost), a Markdown summary, run.json evidence and a ready-to-apply fix.patch.

Why parallel bisection matters: each round shrinks the list of suspects about 9 times instead of 2 times. For 200 commits that's 3 rounds instead of 8. Each round costs the time of one test run, so fewer rounds means a faster answer.

The demo: ₹42.50 becomes ₹42

kirana-billing is a small grocery-billing library that Culprit generates the same way every time: 200 commits, 6 release tags, and a test suite that passes at every commit. Customers report bills ₹1 short: a ₹42.50 bill shows ₹42, while ₹43.50 still shows ₹44.

Culprit finds commit #112, refactor(money): keep amounts in integer paise. That refactor replaced Decimal ROUND_HALF_UP with Python's round(), which rounds .5 to the nearest even number (42.5 → 42, 43.5 → 44). The existing tests only used 43.50, so they still passed. A plain revert fails, because commit #118 added CSV export code that depends on the new to_paise() helper. Culprit's 3-line minimal fix keeps the refactor, rounds 50 paise up, and passes both the regression test and the full suite.

Our recorded offline run took 2 bisect rounds after the scout, 33 isolated test runs and 24.7 seconds from start to finish.

How we built it

Culprit is a Python CLI with four commands: culprit hunt, culprit demo, culprit doctor and culprit report.

Model (NVIDIA Nemotron via Nebius Token Factory)	Job
Nemotron 3 Nano (`nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B`)	Picks which files to read
Nemotron 3 Super (`nvidia/nemotron-3-super-120b-a12b`)	Writes the reproduction test and the fixes; backup explainer
Nemotron 3 Ultra (`nvidia/Nemotron-3-Ultra-550b-a55b`)	Explains the root cause

Nebius Token Factory: Culprit uses its OpenAI-compatible endpoint. Every reply must be strict JSON, with a limited retry and a fallback when JSON mode isn't supported. Calls, tokens, time and estimated cost are tracked per model.
Nebius Sandboxes (contree-sdk 0.3.6): Culprit uploads the repo once as a git bundle, installs its dependencies in python:3.12-slim, and saves that state as a checkpoint. Every probe and every fix check is a throwaway branch of that checkpoint, 8 at a time by default (configurable from 1 to 40). The code being debugged never runs on your machine.
Same scripts everywhere: the setup, probe and verify shell scripts run unchanged in Sandboxes and in the local runner, so the offline demo exercises the same logic.
Patches built with plain git in temporary clones, never copied from model text. File paths from the model are validated, and symlinks or attempts to overwrite the regression test are rejected.
Safety: keys come only from environment variables or .env, and are hidden in events and reports. Ctrl+C stops running work cleanly and still writes a partial report.
Output: a Rich live terminal view whose history strip fills in round by round, and a self-contained HTML report that also works on mobile.
165 automated tests, including a checker that runs the demo repo's own suite at all 200 commits.

Status

The full pipeline is built and tested. Our video uses Culprit's offline mode: real git, real pytest, real parallel bisection and real fix checks on the bundled demo, with scripted stand-ins in place of the model replies. The live path (Nemotron via Token Factory, plus Nebius Sandboxes) is implemented and covered by tests that simulate the Nebius services. Our Nebius credentials weren't ready in time to record a live run before the deadline. With a key in .env, culprit doctor and culprit demo run it live.

Try it (no keys needed)

git clone https://github.com/DrCatnip/culprit && cd culprit
python3 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
culprit demo --offline

Challenges we ran into

Tests that mislead. A test written for today's code often breaks on old code for unrelated reasons, like missing functions. Treating that as "bug present" blames the wrong commit. So we built PASS/FAIL/SKIP rules that compare how the test fails.
Trusting model output. Every model reply must be strict JSON, and every claim is checked by running code. The test must fail and pass in the right places, and the fix must pass the regression test and the suite.
A fair demo. We needed a realistic 200-commit history where the bug is subtle, every commit's own tests pass, and the obvious fix (a revert) is wrong.
Sandbox output limits. Output is cut off at about 64 KB. So the verify script prints exit codes before the logs, and we build patches on the host with git instead of reading them from logs.
Credentials and time. We didn't get live Nebius access set up within our build window. So we invested in an offline mode that runs the same pipeline end to end.

Accomplishments that we're proud of

An agent that doesn't just suggest a fix. It proves the fix with a new regression test and the full suite, and shows its work in one shareable report.
In the demo, Culprit picks the targeted fix over the revert, for the right reason.
165 tests, a demo anyone can reproduce in a minute, and careful handling of edge cases: flaky results, commits that can't be tested, revert conflicts, Ctrl+C.

What we learned

In debugging, the number of rounds is what costs time, not the number of commits. That's why parallel testing helps so much.
Agents become reliable when every step is checked by running code.
Clear result labels (PASS / FAIL / SKIP) matter more than clever prompts.

What's next

Record live Nemotron + Sandbox runs and publish real timing and cost numbers.
A GitHub Action that starts a hunt when someone files a bug.
More languages and test frameworks, and better handling of merges and flaky tests.