PatchPilot

An autonomous coding agent that takes a GitHub bug to a verified pull request, powered by NVIDIA Nemotron on Nebius.

Comment

Inspiration

For the Coding & Agentic Engineering Track, we asked: what if an AI agent didn't just suggest a fix, but proved it? Most AI coding tools stop at "here's a patch." Developers still have to check that it compiles, passes the tests, and fixes the bug. With NVIDIA Nemotron on Nebius Token Factory powering a planner, a worker, and an independent reviewer, we built PatchPilot: an agent that investigates the code, patches it, tests it in a sandbox, repairs its own failures, and only opens a pull request once the fix is verified.

What it does

PatchPilot is an autonomous coding agent powered by NVIDIA Nemotron models served through Nebius Token Factory. You give it a GitHub repository and a bug, and it takes the bug all the way to a verified pull request:

  1. Sign in with GitHub and connect a repository. PatchPilot clones it, scans it, and builds a repo map: stack, routes, tests, and files.
  2. Ask. Ask questions about the code or describe a suspected bug. Ask answers from the repository itself and ends with a verdict: bug confirmed (with a one-line summary) or no bug.
  3. Fix. A confirmed bug is handed to Fix with Ask's findings. PatchPilot drafts a success checklist (a verification contract) for that issue. The agent then plans, edits the code, and runs the repo's own install, test, lint, and build commands inside a Docker sandbox. If something fails, it repairs its own attempt and tries again.
  4. Independent review. A separate reviewer agent checks the diff against the checklist.
  5. Pull request. PatchPilot opens a PR as the signed-in user, forking the repo first if needed.

A patch is only marked verified when every required checklist item passes and the reviewer approves:

Otherwise the run ends as "needs review" and the UI shows exactly why: which check failed, the reviewer's findings, and the reason each attempt was rejected.

How we built it

  • Frontend: Next.js (App Router, TypeScript) and Tailwind CSS, hosted on Vercel. Next.js forwards /api/* to the backend, so the session cookie stays first-party.
  • Backend: FastAPI (Python 3.11+), httpx, and pydantic-settings, running on AWS Lightsail behind Nginx with Let's Encrypt HTTPS and managed by systemd.
  • Models: NVIDIA Nemotron on Nebius Token Factory through an OpenAI-compatible API. A planner model writes the plan and a worker model drives the tool loop.
  • Agent loop: the model calls tools (list_files, read_file, replace_code, git_diff, run_command, …). We execute each call, send the result back, and repeat until the model finishes. Hard limits on steps, time, and repeated calls keep the agent from looping forever.
  • Code understanding: tree-sitter parses JS/TS for the repo map, and keyword ranking points the agent at the most likely files first.
  • Verification: allow-listed commands only, run in Docker containers. A baseline test run before any edit means pre-existing failures aren't blamed on the patch.
  • GitHub: OAuth sign-in, cloning, branch or fork, and pull requests through the GitHub API.

Challenges we ran into

  • Tool calling across providers. Not every model endpoint accepts native tool calls, so we built an automatic fallback to a strict JSON protocol. Models also sometimes call tools that don't exist (for example str_replace_editor). We return a clear error so the agent recovers instead of crashing.
  • Agents that get stuck. We added a guard that stops the run when the same tool call keeps returning the same result, plus step and time budgets per attempt.
  • "Tests pass" isn't "fixed." Early fixes passed the tests but skipped lint, the build, or a regression test. That led to the verification contract, the independent reviewer, and retry attempts that learn from earlier failures.
  • Cross-site cookies. With the frontend on Vercel and the backend on a separate host, cookies broke. Routing /api/* through the frontend kept sessions first-party, with no relaxed cookie settings.
  • Native library crashes. One tree-sitter release corrupted the heap on Windows while mapping real repos, so we pinned a known-good version.

Accomplishments that we're proud of

  • An end-to-end flow that works on real repos: Ask → confirmed bug → checklist → patch → sandboxed tests, lint and build → independent review → GitHub pull request.
  • The agent catches problems beyond the one it was asked about. On our demo shop repo it fixed the reported total() bug (price + qty → price * qty) and a related quantity-merge bug the tests exposed.
  • Honest results: a run that can't prove its fix ends as "needs review" with the reasons shown, never as a false "verified."

What we learned

  • Verification is the product. Generating a patch is the easy part; proving it is correct is where the value is.
  • Small, explicit contracts (a checklist per issue) make agent output much easier to trust and to debug.
  • Deployment details like memory, permissions, and cookies can quietly break an AI system, so observability and clear error messages matter as much as the model.
  • Splitting the work across a planner, a worker, and a reviewer gives better results than one model doing everything.

What's next for PatchPilot

  • Private repositories through GitHub App installs.
  • Persistent storage for sessions, repos, and runs, so a restart no longer signs everyone out.
  • Smarter retrieval: ripgrep-powered search and LLM re-ranking of relevant files.
  • More languages, starting with full Python code mapping.
  • Live progress streaming and a "stuck run" warning in the UI.

Built With

Share this project:

Updates

Submission history