ARCHON

Autonomous, sandbox-verified software repair and AI-stack migration, running on Nebius Token Factory and NVIDIA Nemotron.

Comment

Inspiration

A dependency ships a breaking release. The test suite goes red. Someone reads the traceback, finds the changelog, patches it, re-runs the tests. Most of that loop isn't creative work — it's lookup and iteration. Pasting the stack trace into a chat model doesn't close the loop either: the model's training data predates the break, it suggests imports that no longer exist, and it never sees whether its own suggestion actually worked. Letting an agent run code to find out is a real safety problem on a laptop or shared CI runner — which is why most "AI coding agents" stay chat assistants that never execute anything.

We wanted an agent that closes the loop for real: reproduces the failure, researches it, patches it, and proves the patch works, in an environment safe enough to let it run unsupervised.

What it does

ARCHON takes a repository with a failing test suite, reproduces the failure inside a Nebius Token Factory Sandbox, researches the error with the Tavily Search API, asks NVIDIA Nemotron 3 Ultra to diagnose and patch it, and re-runs the tests until they pass. Every candidate patch runs on its own sandbox fork — losing attempts are simply never referenced again, no rollback needed, because nothing shared was ever mutated.

Two mission types share that loop:

  • Bug healing — point it at a repo and a failing test command, or a SWE-bench Verified instance, and it returns a verified diff.
  • AI-stack migration — point it at a repo calling the OpenAI or Anthropic SDK, and it rewrites the client to Nebius Token Factory with Nemotron models, verified by the repo's own tests plus a parity harness, with a cost estimate from a dated price table.

Every step — reasoning, Tavily queries, sandbox terminal output, the diff — streams live to a web cockpit.

How we built it

  • Orchestrator: FastAPI, driving a deterministic mission state machine (ADR-008 — the supervisor is code, not a model, so control flow is auditable).
  • Model routing by tier: Nemotron 3 Ultra (550B-A55B) does the hard reasoning — root-cause analysis and patch synthesis. Nemotron 3 Super (120B-A12B) supervises, reviews, and synthesizes Tavily queries. Nemotron 3 Nano (30B-A3B) compacts test logs. All three run through Nebius Token Factory's OpenAI-compatible API.
  • Execution: Token Factory Sandboxes (contree-sdk) — clone, install, run tests, fork one sandbox per candidate patch.
  • Grounding: the Tavily Search API is called inside the reasoning loop, before the first patch attempt, with results shown as untrusted context in the reasoning stream.
  • Cockpit: Next.js 16 client with an SSE reasoning stream, an xterm.js sandbox terminal (one tab per fork), and a Monaco diff viewer.
  • Judging: for SWE-bench missions, resolution is judged strictly on the instance's FAIL_TO_PASS/PASS_TO_PASS test ids, never on process exit code, because the dataset contains truncated ids and network-dependent unrelated tests (ADR-013/014). Candidate patches are exact search-and-replace edits applied by the executor, with git diff producing the diff — not model-written unified diffs, which don't apply reliably (ADR-012).
  • Public demo: a stateless replay server on Vercel (ADR-016) streams nine recorded golden-dataset missions with the same REST+SSE contract as the live orchestrator, so judges can watch full runs with zero credentials and zero spend.

Challenges we ran into

The most important bug we found was in our own verification, not the agent's patches. Our first live migration missions both came back "VERIFIED" — but the patches never actually touched the client: one was a 2-line test-name change, the other swapped a dependency in pyproject.toml. Mocked tests stayed green because the client was never rewritten. We fixed this with a structural gate (ADR-015): a migration only resolves when a post-patch scan finds zero closed-provider client constructions or model literals left in source — green tests alone prove nothing.

We also hit a real limitation we're documenting rather than hiding: a subproject buried inside a monorepo resolved 0 of 3 times in Sandboxes, while the identical project as a standalone repo resolved first try — Ultra tends to paraphrase source files in that prompt shape. And one migration mission, anthropic_summarizer, failed honestly after 8 iterations: the full Messages-API-to-chat-completions translation, plus its Anthropic-shaped test doubles, didn't land inside budget, and our completeness gate correctly refused every shallow pass.

Accomplishments that we're proud of

Live, verified results, not projections: 6 of 6 SWE-bench Verified psf/requests instances resolved inside real Token Factory Sandboxes for $0.67 of inference total. psf__requests-1142 specifically: one iteration, 49 seconds, $0.032. A live migration mission (openai_chat_service) verified in 1 iteration for $0.016 — with a gate that genuinely proves the closed-provider code is gone, not just that mocked tests pass.

What we learned

That the safety net matters more than the model. Nemotron produced good patches quickly; what made the results trustworthy was the deterministic supervisor, the strict SWE-bench judging criteria, and the structural migration-completeness gate that caught our own false positives before they became a demo we'd have been embarrassed to show.

What's next for ARCHON

Wider SWE-bench coverage beyond psf/requests, a fix for the monorepo-subdirectory paraphrasing failure mode, and a full Anthropic Messages-API migration path that can land anthropic_summarizer-shaped missions inside budget.

Built With

  • contree-sdk
  • fastapi
  • langsmith
  • monaco-editor
  • nebius-token-factory
  • nextjs
  • nvidia-nemotron
  • pytest
  • python
  • server-sent-events
  • swe-bench
  • tavily-api
  • typescript
  • vercel
  • xterm-js
Share this project:

Updates

Submission history