Apprentice

Our hack: $1

This is a hackathon, so we gave our project a hack of its own: prove that Apprentice works on a one dollar budget. No big credit grants, no hourly GPUs: one dollar of Nebius Token Factory trial credit for everything, with spending capped in code.

That one constraint shaped everything you will see:

• The live specialist teaches through instructions. Token Factory serves a fine-tuned model only from an hourly dedicated endpoint, which a dollar cannot buy. So the shipped specialist, NVIDIA Nemotron 3 Nano, is taught through a playbook and served per token.
• The fine-tune is small and its adapter is tested on our own CPU. For the same reason, the Qwen3 4B LoRA fine-tune was sized to fit, and we downloaded its adapter to score it ourselves.
• Reasoning is switched off for short answers. Nemotron's thinking would otherwise eat tokens on one-word labels.
• Every call is priced before it is sent.

Three full live teaching runs, a LoRA fine-tune and every test came to $0.29. Nothing was charged to a card.

Inspiration

Most of what teams ask a large model to do every day is one narrow job: route a ticket, tag a document, pick a category. Frontier models do these jobs well and cost a lot at volume; small models are cheap and miss. Closing that gap by hand means writing training data, fine-tuning, and hoping it helped. We wanted an agent that does the whole job and proves the result. And if it can be done for under a dollar, any team can afford to try it on their own narrow jobs.

What it does

Give Apprentice a narrow job and labelled examples. It runs a six-stage loop, live on Token Factory, that you can watch:

1. Intake splits the examples into train, dev and a held-out test.
2. Baseline scores the small model, NVIDIA Nemotron 3 Nano, on the held-out test, and scores Nemotron 3 Ultra on the same test as the bar to chase.
3. Teach: Nemotron 3 Super groups Nano's mistakes into failure types. Nemotron 3 Ultra writes new cases aimed at each one. Super labels every case blind, and only the cases both agree on are kept.
4. Train writes a playbook (house rules and worked examples) into Nano's instructions, or runs a LoRA fine-tune on Token Factory.
5. Prove runs the same held-out test again.
6. Ship happens only if the taught model gets more tickets right than the base.

Every call is shown with its tokens, time and cost.

Results on 120 held-out support tickets (two thirds of them hard: mixed intents, misleading keywords, house routing rules):

• Nemotron 3 Nano went from 74% to 87% in our best live run, four tickets short of Nemotron 3 Ultra's 90%, at $0.037 per 1,000 tickets against Ultra's $0.196.
• Across three live runs it gained 11, 3 and 15 tickets (73% to 81% on average). We show all three in the app, the film and the results, including the weak one.
• In weights mode, a LoRA fine-tune of Qwen3 4B on Token Factory took it from 74.2% to 76.7%. The adapter is published.

How we built it

• A shared Token Factory core (packages/nebius-core): an OpenAI-compatible client for chat, files, fine-tuning jobs and downloads; call telemetry; a stub transport so the whole app runs offline; and a spend guard.
• The spend guard prices every call from Token Factory's own model list, reserves the worst case before sending, writes the real cost from the returned usage to a ledger, and halts on any billing error. It refuses fine-tuning without an approved quote and refuses dedicated endpoints.
• The loop (src/server): a deterministic eval harness (temperature 0, fixed prompt, strict label reading) and teaching through Nemotron. Every run's prompts, cases, evals and calls are saved, so it can be audited. The web app streams progress live over server-sent events.
• Weights mode: training files built from the agreed teacher cases, a guarded LoRA job on Token Factory, the adapter downloaded through the files API, and a CPU harness (PyTorch, Transformers, PEFT) that scores base and tuned with the app's exact prompt.
• A deterministic dataset: 414 support tickets for a fictional notes app, eight queues, with test wording that never appears in training.

Challenges we ran into

• The planned student was not served per token. Llama 3.1 8B is tunable on Token Factory but not served per token, so we switched students on day one.
• No way to test a fine-tune on the platform. Fine-tuned models are served only on dedicated endpoints, which bill by the hour. That led to playbook mode, where the live specialist is a served model with taught instructions, and to downloading the adapter and testing it on CPU.
• Reasoning ate short answers. With thinking on, Nemotron 3 Super spent a 300-token allowance and returned nothing. Turning thinking off gave 2-token answers.
• An 11-hour queue for the fine-tune, with no signal, then 3 minutes of training.
• Downloaded adapters loaded as garbage. The weight names lack the prefix PEFT expects, so PEFT silently kept random weights and the model scored 0 of 120. We found it and fixed it in the harness.
• Run-to-run spread. Teaching depends on which failure types the grader finds. We report every run. We also tested one method change on the dev split only, and kept the test split for final scores.

Accomplishments that we're proud of

• A taught NVIDIA Nemotron 3 Nano that gets within four tickets of Nemotron 3 Ultra at a fifth of the cost, proven on held-out tickets.
• Our hack held: three live runs, a fine-tune and every test for 29 cents of a one-dollar budget, with spending capped in code.
• Honest numbers: every run reported, held-out data guarded by tests, and the method chosen on the dev split.

What we learned

Teaching through instructions is fast, cheap and servable today: a whole run takes about a minute and five cents. A LoRA fine-tune needs enough optimizer steps to matter; with packing, a small dataset makes very few. And the grader's view of the mistakes drives the result more than anything else.

What's next for Apprentice

• Fine-tune Nemotron Nano directly once Token Factory supports it, and serve LoRA adapters per token.
• Tavily-grounded teaching cases, using real customer wording from the web.
• More optimizer steps for weights mode, chosen on the dev split.
• Jobs beyond classification: extraction and short-form answers, scored the same way.

Built With

• elevenlabs
• express.js
• hugging-face-transformers
• lora
• nebius-token-factory
• node.js
• nvidia-nemotron-3-nano
• nvidia-nemotron-3-super
• nvidia-nemotron-3-ultra
• peft
• playwright
• python
• pytorch
• qwen3
• remotion