Forkline

Inspiration

Small businesses have the knowledge an AI support assistant needs: an FAQ, a menu, a return policy. What they don't have is a way to turn it into a model of their own. Fine-tuning still means writing training data by hand, renting GPUs, wiring up a server and guessing whether the result is any better than the base model. And once someone has built a good model, there is no simple way to let others use it and get paid for it.

I wanted one place where a person can go from "here is my FAQ" to "here is a model anyone can call, and it earns money", and where open NVIDIA models running on Nebius are the default, not an afterthought.

What it does

Forkline is a marketplace for fine-tuned Nemotron models.

1. Build data. Paste an FAQ, manual or policy. Nemotron on Nebius Token Factory extracts the facts, writes several questions per fact (long, short, and ones that compare similar facts) and returns chat examples you can review before training. You can also upload your own JSONL.
2. Train. Forkline runs LoRA fine-tuning of NVIDIA Nemotron 3 Nano 4B (or Nemotron 3.5 Lightning 30B-A3B) as a Nebius Serverless AI Job. Part of the data is held out for evaluation.
3. Deploy. The base model plus the adapter is served with vLLM on a Nebius Serverless AI Endpoint. Idle endpoints stop after 15 minutes; the next call starts them again.
4. Evaluate. Each held-out question goes to your model and to the base Nemotron. Nemotron on Token Factory grades both answers against the reference (1-5). Forkline shows average scores, win rate, a sign test and a bootstrap interval, and says so when the sample is too small.
5. Publish and earn. Set a price per input and output token, publish to the store, and anyone can call the model through an OpenAI-compatible API (https://forkline.ai/v1/chat/completions). Callers pay from prepaid credits; the creator earns the price minus a 20% platform fee, paid out through Stripe Connect.

Everything in the demo video runs on the live site, https://forkline.ai, with real Nebius jobs and endpoints.

Demo model: midori-bakery-faq, trained on the FAQ of Midori Bakery, a fictional shop. 231 examples, 6 epochs, about ¥130 (under $1) of GPU time. On 26 held-out short questions it scored 4.96 vs 2.19 against the base Nemotron (24 wins, 2 ties, 0 losses, p < 0.001). It is published in the store, so you can call it yourself.

How I built it

• App: React Router and Hono, served as a Node server on a Nebius CPU VM behind Caddy, TiDB Serverless with Drizzle, better-auth for accounts, Stripe and Stripe Connect for credits and payouts. Japanese and English UI.
• Training: a container with PEFT and TRL (bf16, one GPU) that reads the dataset from Nebius Object Storage and writes the adapter back. Forkline starts it through the Serverless AI Jobs API and polls it from a cron trigger every minute.
• Serving: a vLLM container with --enable-lora, started through the Serverless AI Endpoints API. The gateway checks whether the endpoint is up, wakes it if not, and returns 503 model_warming_up with Retry-After until it is ready.
• Nemotron on Token Factory: builds the training data, answers as the baseline, grades the evaluation and drafts the model card. I use structured output (JSON schema) for every step.
• Billing: every amount is an integer in micro-yen. Training holds 1.5x the estimate and settles the real GPU time. Each API call holds the most it can cost before it runs, so a balance never goes negative.

Challenges I ran into

• Nemotron is a hybrid Mamba-2/Transformer model. My first training job failed because the Mamba kernels were missing; the next one failed on a transformers 5 change. I pinned the training libraries and now test the exact versions on CPU before pushing an image.
• The first model learned almost nothing (loss 2.98 after 15 steps). Training only on the assistant turn and keeping enough optimizer steps for small datasets fixed it.
• One question per fact is not enough. The first data builder asked Nemotron for examples in one shot. It produced about one question per fact, and the held-out split removed some facts from training entirely: 6 of 12 test questions were right. Splitting the work into "extract facts" and "paraphrase each fact", with a required contrasts field for facts that are easy to mix up, brought it to 11 of 12. Instructions alone did not make Nemotron write those items; making them a required field in the JSON schema did.
• My evaluation was too kind. The English model first scored 5.00 vs 2.05, 20-0. Then I asked it "Can I drive there?" and it got it wrong. 85% of the generated questions contained the shop's name, and the held-out questions were made the same way, so the test was easier than real use. I added a field for short questions without the name, split lines that held several facts, and took the evaluation questions from the short ones. The score after retraining (4.96 vs 2.19) is lower, and more honest.
• The control plane was unreachable from Cloudflare Workers. The site started on Cloudflare Workers. Token Factory and the endpoint URLs worked, but every call to the Nebius control-plane API (auth.eu.nebius.com, api.nebius.cloud) ended in a 522. Those addresses are routed through Cloudflare, and a Worker cannot open a connection to them. The cron could not stop an idle endpoint, and it ran for about ten hours before I noticed. I moved the whole site to a Nebius CPU VM: the same code now runs as a Node server behind Caddy, with the cron in-process and email sent through the Cloudflare Email Service API. Since the move, the VM starts an endpoint on the first call and stops it after 15 idle minutes, as designed.
• Endpoints timed out pulling images. Pulling from GitHub Container Registry took about 20 minutes and failed four times in a row. Mirroring the images to Nebius Container Registry in the same region cut the pull to about 3.5 minutes and the cold start to about 9 minutes.

Accomplishments that I'm proud of

• The whole loop works on the live site: paste an FAQ, train, deploy, evaluate against the base model, publish, and call it with an API key.
• A useful model for about ¥132 (under $1) in data building, training and evaluation together.
• The evaluation is built to catch its own weak spots: a sign test, a confidence interval and a warning for small samples, instead of a single score.

What I learned

• For small fine-tunes, how you write the data matters more than the hyperparameters. Many varied questions per fact beat more epochs.
• With Nemotron, a required field in a JSON schema is a much stronger instruction than a sentence in the prompt.
• An automatic evaluation is only as good as its questions. If they are made the same way as the training data, the score will look better than reality.

Known limitations

• The demo model still gets some questions wrong. Out of 12 short questions I asked directly, 10 were right. It made up an answer to "Can I drive there?" and was vague about the way from the station. For "Do you sell sandwiches?", which is not in the FAQ, it once said it could not confirm and once said no.
• A stopped model takes about 9 minutes to start.
• Payments and payouts are in yen through a Japanese Stripe account.

What's next for Forkline

• Better answers to questions outside the source material (training on "I can't confirm that" examples).
• Faster cold starts, and serverless LoRA serving if it becomes available, so many models can share one GPU.
• Evaluation questions written separately from the training data, and a public leaderboard per category.

Built With

• caddy
• drizzle
• hono
• nebius-compute
• nebius-object-storage
• nebius-serverless-ai
• nebius-token-factory
• node.js
• nvidia-nemotron
• peft
• python
• react-router
• stripe
• tidb
• trl
• typescript
• vllm