OpenSwitch

Inspiration

I kept noticing the same thing: everyone talks about open models like NVIDIA Nemotron, but almost nobody actually moves a working app onto them. The API isn't the problem. Nebius Token Factory speaks the same language as OpenAI. The problem is fear. Which of my AI calls will break? Which ones really need a big model? Will it even be cheaper? Nobody wants to find that out in production.

So I built a coding agent that answers those questions with real measurements, and then does the switch itself.

What it does

You point OpenSwitch at your app and run one command:

openswitch migrate ./your-app

Then it works on its own, in seven steps:

1. Scan: finds every OpenAI and Anthropic call in your code.
2. Plan: Nemotron 3 Ultra (thinking on) works out what each call is for and what a good answer looks like, even when the model name lives in a config file.
3. Research: Tavily looks up live prices for your current models and for Nemotron, plus the API differences that matter, with source links.
4. Test: it captures your app's real prompts by running your own code against a fake SDK (no OpenAI or Anthropic key needed). It tries every call on 6 Nemotron setups (Nano, Super and Ultra, each with thinking on and off), checks every answer with your app's own code, and has Ultra grade it. Then it picks the cheapest setup that still does the job.
5. Rewrite: Nemotron Ultra changes the code. A checker verifies every model, setting and config file, then runs the migrated app to make sure it behaves exactly as before.
6. Test and fix: it runs your tests (or writes smoke tests if you have none). If something fails, it works out whether the code or the model is at fault and fixes it.
7. Report: everything lands on a new git branch with a clean diff and a report: cost before and after, every score, which Nemotron did which job, and the cases where it struggled. Your main branch is never touched.

On my demo app (HelpDesk AI: 6 AI functions on GPT-4o, GPT-4o-mini, o3-mini and Claude), the estimated AI cost dropped 97%, all 13 of its own tests still pass, and the whole migration cost the agent about 12 cents.

How I built it

• Python, with the OpenAI SDK pointed at Nebius Token Factory. One endpoint serves every Nemotron size, so testing 6 setups per call is just a loop.
• Every Nemotron call goes through one ledger tagged with its job, so the report shows exactly which model did what:
  • Ultra (thinking): plans, rewrites, debugs and writes tests.
  • Ultra (thinking off): judges.
  • Super: reads search results and writes test inputs.
  • Nano, Super and Ultra: the candidates being measured.
• Tavily runs live during the migration, and every fact it finds keeps its source link.
• Deterministic checks (an AST checker, behaviour replay, the app's own parsing code) sit around the AI, so the agent can't just claim success.

Challenges I ran into

• Unstable decisions. My first judge gave one 0–10 grade per answer, and the savings swung from 21% to 80% between runs. I switched to pass/fail checklists, graded every setup (Ultra included), and added adaptive sampling: when a much cheaper option is close, OpenSwitch doubles the test sample before deciding.
• A judge that couldn't read Chinese. It was receiving escaped \uXXXX text and calling perfect translations gibberish. Found it, fixed it.
• Real prompts aren't where the API call is. In apps with a generic complete() wrapper, the real prompt is built elsewhere. OpenSwitch now finds the right caller and confirms with the call stack that it reaches exactly one AI call.
• An agent that cheated. In one test, the debugger changed the app's return value to satisfy a wrong test it had written itself. I added a behaviour check: a recorded model answer is replayed through the old and new code, and the output must be identical. Now it can't happen.

Accomplishments I'm proud of

• The measurements change real decisions. On refund calculations, Super without thinking scored 1.3/10 and with thinking 9.3/10, while easy tasks like sorting tickets run fine on the smallest Nano. Thinking is a choice per task, and OpenSwitch measures it.
• It works end to end from a fresh clone: install, add a key, one command.
• The report is honest. It shows every score and flags calls that would cost more, instead of hiding them.

What I learned

Turning thinking on or off is as big a cost lever as picking the model size. And an AI agent is only as trustworthy as the deterministic checks around it.

What's next

JavaScript/TypeScript support, streaming and tool-calling calls, running OpenSwitch as a Nebius Serverless Job against any repo URL, and opening the pull request on GitHub automatically.

Platform feedback

• Nebius Token Factory: the OpenAI compatibility was seamless, including response_format: json_object, and enable_thinking worked the same way on Nano, Super and Ultra, which made the 6-setup sweep easy. Two requests: report reasoning tokens separately in usage, so max_tokens can be budgeted precisely, and expose prices in /v1/models, so tools don't have to scrape them.
• Docs: search results still mix the old api.studio.nebius.ai endpoint with api.tokenfactory.nebius.com. A clear "current endpoint" note would help.
• NVIDIA Nemotron: Nano with thinking punching above Super without thinking was the most useful surprise of the project.
• Tavily: advanced search gave usable pricing snippets. A structured "pricing extraction" mode would be great for agents.

Built With

• nebius-token-factory
• nvidia-nemotron
• openai
• pytest
• python
• tavily