PatchScout

From production failure to evidence-backed patch: research, reproduce, fix, and verify autonomously.

Comment

Devpost submission notes

Project name

PatchScout

One-line pitch

A source-grounded autonomous dependency migration agent that researches current upstream docs, proposes the smallest patch with NVIDIA Nemotron, and only calls the migration fixed after the original verification passes in an isolated Nebius sandbox.

What it does

Dependency migrations fail in a dangerous way: the code model often knows an older API, produces a plausible patch, and then confidently claims success. PatchScout changes the contract.

Given a public repository and a real upgrade incident, PatchScout:

  • retrieves current migration guides and changelogs with Tavily;
  • asks NVIDIA Nemotron 3 Super on Nebius Token Factory to reason only from the incident plus retrieved evidence;
  • prepares a minimal, reversible migration;
  • executes on a branchable Token Factory Sandbox checkpoint;
  • reruns the same verification command that reproduced the baseline failure;
  • reports the evidence, diff, test result, rollback point, and confidence.

Why it is different

The model never grades its own work. Research, reasoning, execution, and verification are separate stages with a visible chain of custody.

Built with

  • Nebius Token Factory
  • NVIDIA Nemotron 3 Super 120B A12B
  • Nebius Token Factory Sandboxes / ConTree
  • Tavily
  • Next.js / React / TypeScript

Primary track

Coding & Agentic Engineering

Additional prize fit

Best Use of Tavily

Current implementation status

The web product, real Tavily research, real NVIDIA Nemotron inference, repository inspection, source-authority ranking, structured edits, bounded automatic repair, and Token Factory Sandbox execution are complete. The public Express 5 fixture now passes a real fail-before / pass-after run: baseline npm test exits 1, PatchScout edits src/app.js, and the same npm test exits 0 in a branchable Nebius sandbox checkpoint. Build and lint are green, and CI is included in the repository.

Demo flow

  1. Open PatchScout.
  2. Load the Express 4 鈫?Express 5 wildcard incident.
  3. Run the agent.
  4. Show the current-source evidence.
  5. Show the Nemotron diagnosis and one-line route patch.
  6. Show before/after verification evidence and rollback.
  7. Show the live Token Factory Sandbox proof: baseline exit 1, patched exit 0, same npm test command.

Live demo

https://patchscout-five.vercel.app/

Source code

https://github.com/zhaofeipeter/patchscout

Production verification

The deployed app reproduced the Express 5 fixture baseline failure (exit 1), applied the source-grounded structured edit in a Token Factory Sandbox, and reran the same npm test command to exit 0.

Built With

  • nebius-token-factory
  • next-js
  • nvidia-nemotron
  • react
  • tavily
  • token-factory-sandboxes
  • typescript
  • vercel
Share this project:

Updates

posted an update —

Nebius and NVIDIA tool feedback

Nebius Token Factory

PatchScout uses Nebius Token Factory’s OpenAI-compatible chat completions API to run NVIDIA Nemotron 3 Super 120B A12B for dependency-migration diagnosis and structured code-edit generation. The API interface let us integrate hosted inference into our Next.js/TypeScript application without managing model-serving infrastructure.

Our zero-to-hello-world integration path was configuring the API key, endpoint, and model identifier, then making a live inference request. Moving from that first response to an executable patch required additional application work: supplying real repository context, requesting structured JSON, validating edits, and handling retries. We did not record onboarding time, so we cannot provide a measured time-to-first-response.

We would welcome an end-to-end coding-agent example that covers these steps together, including unusable outputs and executable verification. We would use Token Factory again for hosted inference because it fits our existing application stack and lets us focus on the agent workflow. We did not independently deploy workloads on Nebius AI Cloud, so we cannot assess that service from firsthand use.

Token Factory Sandboxes / ConTree

We used the official contree-client JavaScript SDK to execute repository tests in isolated, branchable sandbox checkpoints. This provided the strongest evidence in PatchScout: on our public Express 5 fixture, the baseline npm test exited 1; after editing src/app.js, the same command exited 0. Checkpoints also supported reversible patch attempts and reuse of a clean dependency-installed environment during bounded repair.

The practical onboarding path was connecting the SDK with sandbox access, executing the baseline, applying a candidate edit, and checking the resulting test exit code. The important integration challenge was long-running execution: sandbox and live-agent runs could exceed one minute, so we raised our application route limit from 60 to 300 seconds. This was an orchestration constraint, not evidence of a sandbox service failure.

A documented reference workflow for durable jobs, polling, stage timeouts, and checkpoint recovery would help developers move beyond a synchronous request. We would use Sandboxes/ConTree again because isolated execution and reversible checkpoints make patch verification credible.

NVIDIA Nemotron

We used nvidia/nemotron-3-super-120b-a12b through Token Factory to reason over the incident, repository context, and current upstream documentation retrieved with Tavily, then propose minimal path/search/replace edits. In the verified Express 5 case, the workflow produced a route edit that passed the original failing test.

Our onboarding used hosted inference rather than a local model deployment. The main challenge appeared after basic inference worked: Nemotron occasionally returned no usable edits. We mitigated this with structured JSON output, explicit repository context, retries on empty edit plans, and a bounded repair pass informed by real test output.

More reliable actionable edit output—and clearer differentiation between “no change needed” and “unable to produce a patch”—would improve this use case. We would continue evaluating Nemotron for source-grounded coding workflows, while retaining executable tests as the acceptance criterion. Our current evidence demonstrates one migration fixture, not broad reliability across arbitrary repositories.

Log in or sign up for Devpost to join the conversation.

Submission history