-
-
GhostFix reproduces the failure, separates model hypotheses from executed evidence, and verifies repairs before declaring success.
-
A deliberately demo-injected wrong patch is executed and rejected after pytest proves it still fails. Evidence, not plausibility, decides.
-
Execution evidence drives Nemotron's correction. The model-generated second candidate passes targeted and regression verification.
-
Full repair trajectory: failed candidate → execution evidence → autonomous correction → passing tests → deterministic Proof of Fix: VERIFIED
-
Frozen 7-case benchmark: 1/7 → 3/7 → 4/7 verified repairs. Small-corpus engineering evidence, not a universal success rate.
GhostFix AI
Find the bug. Fix the bug. Prove the fix.
AI coding assistants can generate convincing patches in seconds. But a convincing patch is not the same thing as a correct repair.
A model can misunderstand the failure, modify the wrong code, introduce a regression, or produce a patch that looks plausible but does not actually work.
We built GhostFix AI around a different principle:
Don't trust the patch. Trust the evidence.
GhostFix is an autonomous software repair agent that reproduces bugs, reasons about their root cause, proposes repairs, executes candidate fixes in isolation, runs targeted and regression tests, and produces a deterministic Proof of Fix only when the required evidence passes.
What it does
GhostFix turns software repair into an evidence-driven pipeline:
Bug report → Reproduction → Diagnosis → Candidate patch → Isolated execution → Verification → Proof of Fix
The language model is responsible for reasoning, but it does not decide whether it succeeded.
The LLM reasons. GhostFix tools act. The sandbox executes. Evidence decides.
GhostFix first reproduces the reported failure. Only after observing the failure does the model diagnose the likely root cause.
That diagnosis is explicitly treated as a MODEL HYPOTHESIS, not proof.
A candidate repair is then applied in an isolated workspace and executed against targeted tests and regression tests. Failed execution can become bounded evidence for a correction attempt.
Only successful execution evidence can produce:
VERIFIED — Proof of Fix
How we built it
GhostFix combines model reasoning with deterministic software infrastructure.
We use NVIDIA Nemotron 3.5 Lightning through Nebius Token Factory for diagnosis and repair reasoning.
Around the model, we built deterministic components for:
- secure source handling
- bounded source-context retrieval
- structured root-cause diagnosis
- constrained patch validation
- isolated Docker execution
- targeted-test verification
- regression verification
- evidence-driven self-correction
- source-integrity checking
- deterministic Proof of Fix generation
- reproducible repair evaluation
The architecture deliberately separates reasoning from authority.
Nemotron can propose what should change. It cannot declare itself correct.
Only execution evidence can do that.
Autonomous self-correction
Our main demonstration shows GhostFix responding to a genuinely failing candidate.
For transparency, Candidate 1 is deliberately demo-injected and is not Nemotron output. It is a safe but intentionally incorrect multiplication patch.
GhostFix does not accept the candidate simply because it looks like valid code.
It executes it.
The targeted test genuinely fails: the implementation returns 6 where 5 was expected.
That real pytest failure becomes structured evidence for the correction step.
Candidate 2 is generated by Nemotron, which restores the correct addition behavior.
Verification starts again from the original approved source, with no human patch editing between attempts.
The targeted tests pass.
The regression suite passes.
Only then does GhostFix produce:
VERIFIED — Proof of Fix
The deterministic Proof of Fix records reproduction evidence, the model hypothesis, the complete repair trajectory, executed verification results, provenance, and integrity information.
This makes the distinction between what the model believes and what execution proves explicit.
Engineering evaluation
We created a frozen seven-case repair corpus containing easy, medium, and harder defects, including a regression trap.
Across three engineering iterations, verified repairs improved from:
- V1: 1/7
- V2: 3/7
- V3: 4/7
In V3, all four executable successful candidates passed both targeted and regression verification.
This is a deliberately small engineering benchmark with one run per version. We report it as engineering evidence, not a universal success rate.
Testing outside our benchmark
We also ran a single frozen, no-retry experiment on open pytest issue #15051.
Using bounded evidence-derived source context, GhostFix localized the crash to the same function and line changed by the maintainer's proposed fix in PR #15055.
However, Nemotron's generated unified diff was malformed.
GhostFix rejected it during patch validation.
No candidate was constructed. No verification tests were executed. No retry was allowed. No repair was claimed.
A post-hoc comparison found that GhostFix's root-cause localization matched the proposed maintainer repair, while its intended repair strategy was directionally similar but behaviorally different.
This experiment is not a successful repair.
Instead, it demonstrates an important property of the architecture:
When the evidence is insufficient, GhostFix can refuse to claim success.
Challenges we ran into
One of the hardest problems was deciding where model intelligence should stop and deterministic infrastructure should begin.
Giving an LLM unrestricted control over source selection, patch application, execution, and success determination would make the system easier to build—but much harder to trust.
Another major challenge was patch representation. The real pytest experiment made this concrete: the model correctly localized the defect and produced coherent repair intent, but serialized it as an invalid unified diff.
GhostFix rejected the malformed patch instead of silently repairing the output and presenting the experiment as a success.
That exposed an important distinction:
Semantic repair reasoning and patch serialization are different problems.
What we're proud of
We're most proud that GhostFix preserves the distinction between model hypothesis and executed evidence.
We built an end-to-end system that can reproduce failures, obtain repair reasoning through Nebius, isolate candidate changes, execute real tests, perform bounded evidence-driven correction, preserve the original source, and generate a deterministic Proof of Fix.
We also built a frozen evaluation corpus and measured engineering iterations instead of relying only on a hand-picked demonstration.
Finally, we tested GhostFix against a real external pytest issue and preserved the failed result rather than rerunning until the model happened to succeed.
For us, that is central to trustworthy autonomous software engineering.
What we learned
The biggest lesson was that verification architecture matters as much as model capability.
Software repair contains several independent failure surfaces:
Diagnosis → Repair intent → Patch representation → Application → Targeted behavior → Regression behavior
Success at one stage does not imply success at the next.
Our external experiment demonstrated this directly: diagnosis could be correct while patch serialization still failed.
We also learned that evidence needs provenance. A model-generated explanation should never be presented with the same authority as an executed test result.
What's next
Our next research direction is structured repair operations.
Instead of asking the model to serialize fragile unified diffs directly:
Model → Structured edit intent → Deterministic source transformation → Deterministic diff → Sandbox verification
This could remove an entire class of mechanical patch-generation failures while preserving GhostFix's evidence-driven architecture.
Structured edits still cannot guarantee semantic correctness.
Execution remains the final authority.
Longer term, GhostFix could become an autonomous repair layer for CI systems: reproducing failures, preparing candidate repairs, verifying them in isolation, and attaching machine-readable Proof of Fix evidence for human review.
Log in or sign up for Devpost to join the conversation.