DevPilot begins by accepting the target repository, bug description, and maximum repair attempts.
Analyzes the codebase to identify relevant files and potential issues.
Establishes the initial project state for comparison after code repair.
Uses external search to gather useful technical context for debugging.
Generates and validates candidate code fixes for the identified issue.
Runs checks again to evaluate the fix and detect possible regressions.
Displays code additions and deletions for transparent review of changes.
Provides scorecard and leaderboard views to review available run results.

DevPilot: Autonomous Software Engineering Agent Native Windows 11 Code Repair Pipeline with AI-Assisted Patching and Automated Verification

Inspiration

A bug rarely arrives at a convenient time. A small regression can send a developer digging through unfamiliar files, rebuilding an environment, tracing the cause, and rerunning tests—only for a fix that looks right to break something else. AI can make writing a patch faster, but generating code is not the same as proving that a repair works. That is the problem we wanted DevPilot to tackle. Instead of stopping at a code suggestion, DevPilot aims to connect the full repair loop: understand the reported issue, inspect the project, propose a focused change, run tests in a separate workspace, and make the outcome available for review. When a repair fails, the test results and code differences should help explain what happened rather than hide the failure behind a confident answer. We built DevPilot around a simple principle: let AI help propose the fix, but use observable test results and change inspection to evaluate it. The goal is to take some of the repetitive work out of debugging while keeping developers informed and in control. For the Nebius x NVIDIA Global AI Hackathon, we have also tested Nebius-hosted inference in the repair-generation workflow, giving us a concrete foundation to keep improving and evaluating the system.

What It Does

DevPilot is a Python-based AI software repair project designed to automate parts of the debugging process.

Describe the issue: Provide a bug description so the system can investigate the problem.
Prepare the codebase: Ingest a local repository or a remote repository, depending on the supported workflow.
Map the project: Analyze the code structure and identify relevant components and potential relationships between features and tests.
Generate a repair: Use an LLM-assisted healing process to propose code changes, with heuristic fallback paths available in the implementation.
Test the changes: Run tests in a separate workspace to evaluate whether the proposed repair works.
Check the result: Inspect the outcome and code differences to help identify unsuccessful changes or possible regressions.
Record results: Use a SQLite-backed leaderboard and reporting components to store benchmark information and present repair results. The central idea is to connect AI-generated code changes with a verification process instead of treating generated code as automatically correct.

How We Built It

DevPilot is organized as a modular Python project, with separate components responsible for different stages of the repair workflow.

Phase 1 — Orchestrator: Coordinates the repair process and provides the orchestration interface.
Phase 2 — Input Pipeline: Prepares the repository or workspace for analysis.
Phase 3 — Feature Mapper: Scans the project structure and helps relate code components to relevant tests.
Phase 4 — Healer: Generates candidate repairs and applies code patches.
Phase 5 — Sandbox Testing: Runs tests in a separate workspace and supports browser-based testing through Playwright where configured.
Phase 6 — Metrics and Diff: Examines code changes and repair outcomes.
Phase 6 — Repair Loop: Contains repair-loop functionality for coordinating repeated attempts.
Phase 7 — Leaderboard: Stores benchmark results using SQLite.
Phase 8 — Tavily Search: Provides an optional web-research component. The project also includes a lightweight HTML, CSS, and JavaScript dashboard. The intended workflow is to prepare the codebase, analyze the issue, generate a repair, test the result, inspect the changes, and record the outcome. The LLM helps with reasoning and patch generation, while ordinary program logic and test execution provide evidence about whether a proposed change works. The project uses Python, FastAPI, Pydantic, HTTPX, SQLite, subprocess-based test execution, and code-analysis techniques. Optional integrations depend on the configured environment and available services. Nebius-hosted inference is connected to the repair-generation workflow through the team's Nebius setup using a supported NVIDIA model. The team reports that the Nebius integration has now been tested successfully and that the relevant tests are passing. This confirms the tested integration path, but does not imply 100% test coverage or guarantee that every repair will succeed. The exact model, test scope, and measured performance should be recorded alongside future validation results.

Challenges We Ran Into

Applying code changes safely AI-generated patches can contain incorrect edits or formatting problems. DevPilot includes a block-based patching approach intended to make code changes more targeted than replacing entire files.
Distinguishing existing failures from new regressions A repository may already have failing tests before a repair begins. Comparing the baseline with the post-repair result is important for distinguishing existing problems from failures introduced by a proposed change.
Running tests in isolated workspaces Testing changes in a separate workspace helps reduce the risk of interfering with the original project. Subprocess management, timeouts, and cleanup also require careful handling.
Making repair attempts useful When a candidate repair fails, the system needs meaningful test or error information to evaluate the outcome and determine whether another attempt is appropriate. Repeated attempts must also be bounded.

Accomplishments We’re Proud Of

Built a modular architecture that separates orchestration, code analysis, repair generation, testing, and reporting.
Developed a patching component for targeted code changes.
Integrated test execution into the repair workflow rather than relying only on model-generated explanations.
Included components for examining code differences and storing benchmark results.
Created the foundation for browser-based testing and a lightweight developer dashboard.
The team reports that the Nebius-hosted inference integration has been tested successfully and that the relevant integration tests are passing.
The latest reported test run also passed all currently configured DevPilot tests. These results apply to the tests actually executed; they do not imply 100% test coverage or guarantee that every repair will succeed.

What We Learned

Generating code is only one part of fixing a bug. A proposed repair needs to be tested and reviewed.
Existing failures matter. A meaningful verification process should consider the state of the project before the change.
Isolation makes experimentation safer. Testing candidate repairs separately helps protect the original codebase.
Automation needs boundaries. Timeouts, retry limits, and failure handling are important when an agent runs commands or modifies code.
Reliable results require evidence. Test outcomes and code differences are more useful than simply claiming that an AI-generated patch is correct.

What's Next for DevPilot

With the Nebius-hosted inference integration tested successfully and the relevant tests passing, our next step is to make DevPilot more measurable and extend its repair workflow to more complex projects.

Build a repeatable repair benchmark: Evaluate DevPilot against a set of known bugs and record baseline test results, generated patches, final outcomes, and execution time. This will help us understand where repairs work, where they fail, and how performance changes over time.
Make verification evidence easier to inspect: Improve regression comparisons and reports so developers can see what changed, which tests ran, and why a repair was accepted, rejected, or left unresolved.
Strengthen the Nebius-powered workflow: Document the exact NVIDIA model and API path used, preserve reproducible test evidence, and measure response quality, latency, and failure cases across a broader set of repair tasks.
Explore an MCP reviewer: Expose selected repair, test, and diff evidence through a Model Context Protocol interface so other developer tools can query the results of a DevPilot run.
Handle multi-repository bugs: Extend project mapping to help trace dependencies between repositories, such as a frontend and the backend API it relies on.
Expand language support carefully: Add languages such as Go, Rust, and Java alongside the analyzers and test runners needed to verify repairs for each ecosystem.
Explore Nebius cloud scaling: Investigate cloud-based worker pools for running independent repair and benchmark tasks in parallel, while preserving clear test records and failure handling.

The aim is not simply to add features. It is to make DevPilot's repair process easier to reproduce, its results easier to review, and its future expansion grounded in evidence from real test runs.

Closing DevPilot explores a practical direction for AI-assisted software engineering: moving beyond code suggestions toward a structured process that attempts repairs and evaluates them through testing. The long-term goal is to help developers spend less time on repetitive debugging work while keeping verification, transparency, and human review central to the repair process.