-
-
Full revival: GPT-5.6 took a dead repo from 0 to 61 passing tests with one self-generated fix, verified by real test results.
-
The Graveyard Benchmark: four real dead repos. One full revival, two honest partials, one correctly declined as out of scope.
-
Multi-step reasoning: GPT-5.6 staged several fixes and self-corrected a malformed diff after our engine fed the error back.
-
The agent cannot cheat: legit source fixes are accepted, while test-file edits, skip-injection, and tampered patches are rejected.
Inspiration
Every developer has hit it: you find the perfect open-source library for your problem, and then you see the last commit was in 2018. It still has stars, still has downloads, still shows up in search results, but it no longer installs. The code was fine. Python moved on, a dependency was removed, an API changed, and the maintainer disappeared. The knowledge to fix it exists, but nobody is there to apply it.
We realized this is a perfect job for an AI agent, but also a dangerous one. An agent that edits code to make tests pass has an obvious failure mode: it can cheat. It can delete the failing test, add a skip marker, or convince itself a broken fix worked. So we set out to build an agent that could revive dead code and could be trusted not to fake it.
What it does
Necromancer is a command-line agent that revives abandoned Python repositories. You point it at a dead repo and it:
- Runs the test suite to establish an evidence-based baseline of exactly what is broken.
- Uses GPT-5.6 to read the real failure tracebacks and diagnose the root cause.
- Has GPT-5.6 propose one minimal source-only patch at a time.
- Applies each patch to a disposable snapshot, re-runs the tests, and promotes the patch only if real test results prove genuine progress.
The core principle is: GPT-5.6 proposes, deterministic code decides. The model never gets to declare its own success. An anti-cheat policy forbids editing test files, pytest config, or injecting skips, so the agent physically cannot fake a passing result.
We benchmarked it on four real abandoned repositories:
| Repo | Outcome | Result |
|---|---|---|
| algorithms | Full revival | 0 to 61 passing tests, one AI-generated fix |
| envoy | Partial (6 of 9) | Multi-step cascade; AI self-corrected a malformed patch |
| vincent | Partial | Modern importlib.resources migration |
| django-rest-swagger | Out of scope | Failure was in a dependency, so the agent correctly declined |
How we built it
The system is a deterministic controller (the Director) wrapped around GPT-5.6 reasoning stages. We built it in this order, deliberately putting the trustworthy infrastructure first:
- A sandboxed test runner that installs each repo in an isolated environment and captures collection errors, per-test results, and tracebacks as structured JSON.
- An evidence-based scoring system that turns test results into a comparable score, with a rule that previously-passing tests can never be lost.
- A patch-apply layer with an anti-cheat policy and preimage hash checks.
- A real GPT-5.6 Surgeon, using the Responses API with strict structured outputs, that reads the failure evidence and generates minimal diffs.
We used Codex as a full engineering partner across thirteen documented sessions, from architecture to implementation. Every session is logged in our repo.
Challenges we ran into
The hardest lesson came from our own anti-cheat policy. We claimed the agent could not edit test files, but Codex audited our code and found that our policy only blocked the tests/ directory, leaving root-level test files editable. Our central trust claim was false, and Codex made us fix it before we could honestly make it.
We also discovered that repos surprise you. envoy looked like a one-line fix but hid a cascade of Python 2 incompatibilities. Our evidence-based engine correctly refused to accept fixes that did not actually improve the results, which is exactly what kept the agent honest.
Accomplishments that we're proud of
- GPT-5.6 autonomously revived a dead repository, taking it from completely broken to 61 passing tests with a single self-generated patch.
- The agent self-corrected: when GPT-5.6 produced a malformed diff, our engine fed the error back and the model fixed its own mistake.
- A benchmark with honest mixed results. One full revival, two partials, and one repo the agent correctly identified as out of its scope. We chose to show what does not work, because that honesty is the point.
What we learned
The most important lesson was architectural: never let the language model be the judge of its own success. Every capability we added was only trustworthy because a deterministic layer verified it against real evidence. We also learned that an AI is at its most useful as a partner that challenges you, not just one that writes code. Codex catching a false claim in our own safety policy was more valuable than any feature it wrote.
What's next for Necromancer
- Wiring the full four-stage pipeline (Coroner, Archaeologist, Surgeon, Historian) into the CLI end to end.
- Expanding beyond in-repo source fixes to dependency modernization and test configuration.
- Support for languages beyond Python, and opening pull requests directly against the revived repositories.
Log in or sign up for Devpost to join the conversation.