Inspiration

Every team wants a cheaper, private model that knows its own domain, but building one is still an engineering project: gather data, write training examples, rent a GPU, fine-tune, quantize or serve, then evaluate honestly. Teams without an ML engineer usually stop at prompting a big model. We wanted that whole post-training loop to start from one sentence, run on Nebius with NVIDIA Nemotron at every step, and come back with its cost and an honest evaluation.

What it does

Orchapod is an open-source platform for the post-training lifecycle of a language model, with two front doors:

The Orchapod Agent. Type one sentence. Nemotron 3 Super, on Nebius Token Factory, drafts a pipeline from the node catalog; the server checks every model, input and cost; a plan card shows the nodes, the models and the estimated dollars and minutes. Press Run: Orchapod executes the plan, streams logs and cost, evaluates the result and writes a report. Template cards run ready-made plans in one click.
The canvas. Drag nodes onto a React Flow canvas. Any plan the Agent writes opens there for editing.

A typical plan: research (Tavily reads live web pages) → synthetic data (Nemotron 3 Super writes grounded Q&A pairs, each with its source passage) → data prep → fine-tune (an Unsloth LoRA of NVIDIA Nemotron Nano 8B on an H100 Nebius Serverless Job) → serve (vLLM on an L40S Serverless Endpoint) or quantize (a 4-bit GGUF on CPU) → evaluate (Nemotron 3 Ultra judges the tuned model against its base in both answer orders). A finished run shows an eval view with every judged sample, a dataset card, a side-by-side playground and each step's cost.

Who it is for: teams that need a model fluent in their own material but have no ML engineer: an analyst desk that wants this quarter's earnings figures (our demo), a compliance team working from its rulebook, a support team working from its docs.

How we built it

NVIDIA Nemotron on Nebius Token Factory makes every model call. Nemotron 3 Super plans (strict json_schema output), writes the synthetic data and writes the run report; Nemotron 3 Ultra judges; Nemotron 3.5 Lightning can filter data. No other model provider is called.
Nebius AI Cloud. Serverless Jobs train the student on an H100; Serverless Endpoints serve base and LoRA side by side with vLLM and are deleted at expiry; Object Storage holds every artifact; a Container Registry in the same project serves node images with no credentials; a Nebius VM hosts the public demo.
Tavily Search and Extract feed the research step. Passages keep their source URLs, so every grounded answer is traceable.
The platform. Every step is a Docker container that reads JSON inputs and writes artifacts, logs and a cost record through one SDK. A FastAPI control plane with a DAG executor runs them, and a Next.js app shows each run live. Every billed call is metered per node, estimated before a run and capped during it.
Where Token Factory accelerated us: a 120B teacher and a 550B judge with no GPUs or serving stack of our own; plans that validate on the first call (7.3 s, $0.0025 in our spike); 300 grounded rows in about 2 minutes for $0.03; 30 judged questions in 85 s for $0.08; per-token prices that let the Agent estimate a run before it starts.

Challenges we ran into

Serving a fine-tune. Token Factory's per-token LoRA deploy now answers 410, so we serve the adapter beside its base on a Nebius Serverless Endpoint instead.
Getting GPU work onto Nebius. Jobs pull images anonymously and ours were private on GitHub, so we moved them to a Nebius Container Registry. AI Cloud creates answered PermissionDenied after billing activation until a CLI re-login.
Synthetic data that invents numbers. On a live finance run about one row in ten stated a figure its source page did not hold. Grounded synthesis now keeps only answers whose figures appear in their passage.
Planner mistakes. Nemotron's plan once paired a CPU-quantized LoRA with the 16 GB full-precision base, more than the CPU host's RAM. The server's review now refuses that plan before anything runs.
A training image that broke on GPU. Mamba-2 wheels broke import unsloth; we dropped them.
Early integration bugs (October). A database update had to be committed before the background task that reads it started, and the container SDK was not on PyPI, so it is now copied into each image.

Accomplishments that we're proud of

Live, end to end on Nebius, from the hosted demo. The NVIDIA golden path (Nemotron Nano 8B trained on an H100 Job, served from an L40S Endpoint, judged by Nemotron 3 Ultra) ran twice: 24.2 and 23.5 min, $1.61 and $1.58 against a $2.66 estimate; 7 wins, 22 ties, 1 loss and 7 wins, 20 ties, 3 losses on 30 held-out questions.
The demo video's run: a Q3 2026 bank-earnings analyst from one sentence: 40 live pages, 550 grounded Q&A pairs, a LoRA fine-tune on an H100 Job, a 4.92 GB q4_k_m GGUF on CPU, and 15 wins, 27 ties, 12 losses against the base on 54 held-out questions, in 51 minutes for $1.36.
An evaluation you can trust. Base and tuned get identical prompts and decoding; the judge sees the question, the reference and the source passage and scores both answer orders; a win counts only if it holds in both. We publish losses and caveats as they are.
Public and reproducible. The hosted demo at app.orchapod.dev, MIT code, and make check E2E=1, which runs the three canonical intents and the golden path on deterministic stubs with no key, for $0.

What we learned

Fine-tuning moves style, format and vocabulary more than it injects facts: our tuned models answer from their domain where the base declines, but can misremember figures, so every run is judged before anyone relies on it.
Judging in both answer orders matters: only 0.67 to 0.74 of verdicts held in both orders across our live runs.
Strict JSON schemas make an LLM planner dependable, and a server-side review catches what a schema cannot express.
Cost has to be a first-class output: estimates before Run and caps during it made it safe to open a public demo on real infrastructure.

What's next for Orchapod

Serve fine-tunes per token on Token Factory once LoRA deploy or custom weights are open, instead of a dedicated endpoint.
Evaluate on held-out sources and public benchmarks, not only held-out rows of the same synthetic set.
Pair fine-tuning with retrieval, so figures come from documents while the model learns the domain.
Team workspaces with a shared registry of models and datasets.

Try it

Hosted demo, no login: https://app.orchapod.dev/agent. Press Run on "Synthetic movie likes, shares and subscribes" (about a minute, under a cent), or type a sentence and press Plan. Visitors' runs are capped and GPU steps are admin-only; the finished fine-tune runs are public: 6bf36882 (the video's), 0ee1e20a and d20d30f3.
Code (MIT): https://github.com/OrchapodAI/orchapod