Inspiration
AI coding tools can suggest a fix in seconds, but a plausible patch is not proof that a bug is resolved.
We built RepoDoctor to make automated repairs easier to trust. It investigates a reported issue, reproduces the failure, tests competing fixes, and checks the selected patch in a fresh sandbox—with evidence you can inspect.
Diagnose bugs. Test competing fixes. Prove the repair.
What it does
Give RepoDoctor a GitHub repository, an issue description, and an optional failing command. It runs an engineering workflow:
- Analyze the repository and identify relevant source files.
- Reproduce the reported failure in an isolated sandbox.
- Research technical questions with Tavily when external documentation would help.
- Investigate three hypotheses through distinguishing experiments.
- Compare candidate patches against the same checks.
- Verify the selected patch from a fresh checkout of the original commit.
- Deliver a downloadable patch, regression tests, and an engineering report.
The dashboard exposes commands, sandbox output, experiments, patch evaluations, and verification results.
RepoDoctor distinguishes Verified, Likely Fixed, Inconclusive, and Failed outcomes. Its scoring and verification rules determine the result; the model cannot promote its own answer to “Verified.”
How we built it
- NVIDIA Nemotron, served through Nebius Token Factory, powers repository analysis, hypothesis generation, test generation, patch authoring, and reporting.
- Nebius Token Factory Sandboxes provide isolated execution environments, checkpoints for comparable experiments, and fresh environments for final verification.
- Tavily supplies external documentation when the research agent identifies a useful question.
- Django and PostgreSQL power the application, evidence storage, and durable repair queue.
- Docker Compose runs the web application, worker, and database.
- Pydantic schemas and deterministic validation constrain model responses before they become executable artifacts.
The current configuration uses Nemotron 3 Super 120B-A12B.
Production patches use exact source replacements. Generated test harnesses live separately under .repodoctor/, and repair attempts are restricted to approved files.
Challenges we ran into
The hardest part was making generated code produce trustworthy evidence.
A generated test can fail because of broken imports or shell quoting rather than the reported bug. It can also parse successfully while doing nothing—for example, when the model returns a JSON wrapper instead of executable Python.
We added:
- Validation for generated Python commands and source files.
- Automatic repository import setup for generated Python harnesses.
- Rejection of inert literal-only test files.
- Restricted harness repairs that preserve the command and expected failure marker.
- Baseline checks that confirm a regression test fails before the patch.
- Smaller hypothesis requests and bounded retries for truncated model output.
We also evaluated Nemotron 3 Ultra. Although account access and structured output worked, it repeatedly exhausted output limits on this workflow. We returned to Super based on the observed results.
Accomplishments that we're proud of
RepoDoctor completed a live repair of viraja1/mitm_postman, where captured PATCH requests lost their JSON bodies when exported to Postman.
It generated the one-line production fix:
- if flow.request.method in ['POST', 'PUT']:
+ if flow.request.method in ['POST', 'PUT', 'PATCH']:
The run confirmed that the regression test failed on the original code, then passed reproduction, targeted, and regression checks in a fresh sandbox after applying the patch.
The result was Likely Fixed, because the repository lacked a full test suite. Preserving that distinction is central to RepoDoctor: it reports what the evidence supports.
Our application test suite also reached 94 passing tests.
What we learned
Reliable automated repair needs more than a capable model. It needs controlled execution, useful feedback, protected test criteria, and explicit confidence limits.
We learned that:
- A failing test is useful only when it fails for the relevant reason.
- A passing test needs meaningful assertions.
- Smaller, structured generation tasks can be more manageable than large responses.
- A larger model does not automatically perform better in a particular workflow.
- Fresh-checkout verification provides stronger evidence than repeatedly testing a modified workspace.
What's next for RepoDoctor
- Broader support for repository languages and test frameworks.
- Better test discovery and diagnosis of environment failures.
- Model evaluations based on repair completion, cost, and evidence quality.
- GitHub pull-request creation with attached verification evidence.
- Clearer coverage reporting for repositories without existing test suites.
Our goal is to make every automated repair come with an inspectable explanation of what failed, what changed, and what was actually verified.
Built With
- django
- docker
- docker-compose
- html
- javascript
- nebius-sandboxes
- nebius-token-factory
- nvidia-nemotron
- postgresql
- pydantic
- python
- tavily
Log in or sign up for Devpost to join the conversation.