Upgrade Pilot

The problem

Upgrading a major dependency often breaks code that was working. Fixing it means reading release notes, guessing what changed, and re-running tests until they pass. It is slow, repetitive work, so teams put it off and upgrades pile up.

What it does

Upgrade Pilot takes a Python project and a dependency upgrade and works the way a careful engineer would:

1. It runs the project's tests against the new version in an isolated sandbox to see exactly what breaks.
2. NVIDIA Nemotron 3 Ultra plans the migration from the failing tests.
3. It searches official documentation and migration guides with Tavily, using queries built from the actual errors. Nemotron Nano condenses the results into short migration notes with source links.
4. Nemotron 3 Super patches the application code, then the tests run again. This repeats on the remaining failures until they pass or a limit is reached.
5. Ultra reviews the final diff for behavior changes the tests might not catch and writes a PR-style summary listing the sources used.

The agent edits application code only, never the tests, so a passing result means the project's existing tests really pass. The public demo includes two real breaking upgrades: Pydantic v1 to v2 and SQLAlchemy 1.4 to 2.0. The interface streams every step live, links the documentation sources the agent used, and shows tokens and estimated cost per model tier.

How we built it

• Nebius Token Factory and NVIDIA Nemotron. All model calls go through one OpenAI-compatible Token Factory endpoint, which let us route work by cost and difficulty: Ultra for planning and review, Super for code edits, Nano for high-volume summarization. A usage panel shows the split per run.
• Tavily. Every run makes real Tavily searches built from the failing tests and the dependency being upgraded. Results are summarized and shown in the UI with their sources, so each fix is grounded in current documentation and not in what a model remembers. (add: "restricted to official documentation domains where possible" only if implemented)
• Sandbox. Each run executes pytest in a fresh virtual environment installed from a prebuilt local wheelhouse, so runs are reproducible and do not depend on live package indexes.
• Guardrails. Patches are confined to the run directory, test files cannot be modified, retrieved web text is treated as untrusted reference material and never as instructions, and each run has iteration and token caps. The hosted demo adds per-IP rate limits and a daily spend cap, with recorded real runs as a fallback.
• Stack. Python, FastAPI, server-sent events for live streaming, pytest, Docker, deployed on [host].

Challenges

Getting reliable structured edits out of the model was the hardest part, so malformed output goes through a repair-and-retry step. Keeping the agent honest was the second: it must fix the code and never loosen or edit the tests, so that rule is enforced in code and not left to the prompt. Reproducibility was the third, which is why the sandbox installs from a local wheelhouse.

Accomplishments

A working end-to-end agent on two real breaking upgrades, with live visibility into every step, cited documentation sources, and per-tier cost reporting. [Add measured results only if you have them, e.g. "X of Y runs reached all tests passing".]

What we learned

Splitting the work across Nemotron tiers kept the loop responsive and the cost visible. Grounding the agent in current documentation through search mattered because migration guides change between releases.

What's next

Support for more ecosystems such as npm, opening pull requests directly on GitHub, and running arbitrary user repositories in per-run isolated sandboxes on Nebius serverless compute.

Built With

• docker
• fastapi
• nebius-token-factory
• nvidia-nemotron
• pytest
• python
• tavily