Inspiration

We first built ModelForge at a hackathon in June 2025. The goal was to let people who aren't ML engineers fine-tune a HuggingFace model on their own GPU without writing any training code and democratize access to personalized AI forever. Our mission was to provide everyone in the world the equal opportunity to experiment with AI freely and fully leverage this technology without needing a PhD in Machine Learning. v1 proved the idea worked, and real users (23,000 of them!) then showed us its limits. It had three fixed tasks and one recipe: LoRA with 4-bit quantization. It needed an NVIDIA card, and you had to set up the backend and frontend by hand. Over 2.x and 3.x we added strategies and providers, but the architecture had been built under hackathon time pressure, and every new feature made it more fragile.

For v4 we started over. We asked what the tool should look like if it were built correctly from the beginning: one engine, honest hardware planning, and an interface for every place people actually train models.

What it does

ModelForge v4 is a fine-tuning toolkit that plans around your hardware. You tell it what you want, for example "I want an instruction-following model trained with preference data." It detects your hardware (CUDA GPUs, Apple Silicon, RAM, CPU) and recommends a model, a training method and a configuration that will fit - ALL WITHOUT AN LLM (using our custom-written recommendation calculation engine)!!! Then ModelForge runs the training, streams progress, and gives you a playground to test the result.

  • Four use cases: Instruction Following, Text Generation, Summarization (on a native encoder-decoder trainer) and Extractive QA (with a proper QA head and sliding windows over long passages).
  • Three training objectives: supervised (SFT), preference (DPO) and reinforcement (GRPO, with a reward based on exact-match reference answers).
  • Automatic method selection: for the model you pick, the engine chooses between full fine-tuning, LoRA, 8-bit QLoRA and 4-bit QLoRA, and prefers less quantization when it fits. If you switch models, it recalculates the method, learning rate, adapter settings and parallelism, and Start stays disabled until the new plan is ready.
  • Multi-GPU: data-parallel and FSDP plans, checked against the smallest GPU you selected.
  • Three interfaces on one engine: a browser UI (modelforge run), a terminal wizard (modelforge cli) and a notebook interface with no widgets (modelforge-nb) for Colab or Jupyter.
  • Run management: live progress and ETA over WebSockets, checkpoints you can resume from, run history in SQLite, a separate TensorBoard log per run, and push-to-Hub.
  • Session playground: streaming chat with your fine-tuned model. It uses the model's own chat template, plans memory across GPU, CPU and disk, and refuses to load a model while training is running so it can't take the trainer's VRAM.
  • Built-in docs: the documentation site is part of the app.

Accomplishments that we're proud of

How v4 compares with v1:

ModelForge v1 ModelForge v4
Training methods LoRA + 4-bit only Full FT, LoRA, 8-bit and 4-bit QLoRA, chosen per model
Objectives Supervised only SFT, DPO, GRPO
Hardware Single NVIDIA GPU CUDA, Apple Silicon (MPS), multi-GPU DDP/FSDP
Interfaces Web UI Web UI, terminal wizard, notebook
Install Separate backend/frontend setup One pip install, UI included
Training visibility Basic Live WebSocket progress and ETA, TensorBoard per run, checkpoints and resume
Testing your model One-shot inference Streaming playground with sessions and memory planning
Security Open local server Startup token on every route
Quality Minimal tests ~300 tests, CI for tests, frontend and wheel install
  • Every recommendation can be changed, and changing the model recalculates the plan.
  • The browser, terminal and notebook interfaces share one engine.
  • The docs describe the limits honestly. For example, they say the GRPO reward only checks exact answers, and that a chat template doesn't make an untrained model good at language.

How we built it

  • Backend: Python 3.13, FastAPI and Uvicorn, on Transformers 5, TRL, PEFT, Accelerate and bitsandbytes. Liger kernels are an optional extra for lower VRAM use on CUDA.
  • Architecture: registries and factories for strategies (SFT, DPO, GRPO, QA), tasks (causal LM, seq2seq, QA, token classification), providers and use cases, with strategies and tasks discovered automatically. A single workflow layer (service, job manager, training, playground) is shared by the web, CLI and notebook interfaces, so all three run the same code.
  • Recommendation engine: maps use case, objective and detected hardware to a recommendation validated by Pydantic. Memory estimates are deliberately conservative.
  • Data pipeline: one normalizer for every interface. It renames column aliases before validation (and rejects a dataset that has both an alias and the canonical column), converts prompts and preference pairs to role/content chat messages, and saves the tokenizer and chat template with every output, adapter-only runs included.
  • Frontend: React and Vite, built from a locked package-lock.json and bundled into the wheel. pip install modelforge-finetuning gives you the full UI without Node installed.
  • Security: the local server requires a random startup token on every API and WebSocket route.
  • Quality: about 300 unit tests in 29 files, and a CPU-only GitHub Actions pipeline that runs the Python tests, builds the frontend, builds a wheel from a clean tree, installs it in a new virtualenv, and checks that /, /app and deep links load.

Challenges we ran into

  • Telling the truth about hardware. Earlier versions were too optimistic, and a too-optimistic estimate means a crash hours into training. We rewrote capacity planning so system RAM never counts as extra VRAM, adapter replicas must fit the smallest selected GPU, and estimates are treated as limits rather than promises.
  • Chat templates. Prompts, DPO pairs and GRPO prompts all had to go through the tokenizer's chat template the same way in training and in the playground, without duplicating special tokens. Tokenizers with no template needed a sensible fallback.
  • Preference data in many formats. DPO datasets come with explicit prompts, full conversations, or just chosen/rejected pairs. We had to infer the shared prompt reliably and reject inputs that conflict.
  • Library breakage. Transformers 5 silently ignores logging_dir, so TensorBoard logs went missing until we configured the callback directly.
  • Packaging. Shipping a Python wheel that contains a built React app, and making sure CI never publishes a wheel without the UI, took several rounds of fixes (disk exhaustion in CI, finding console scripts inside a virtualenv, editable installs).
  • Training and inference running together. Letting people use the playground during a training run, without risking the trainer's VRAM, required a GPU-exclusion rule and a memory plan across CPU and disk.

What we learned

  • Rewrite with discipline. Three major versions of adding features to a hackathon base showed us that the architecture caps what you can build. Clear boundaries between strategies, tasks, providers and the workflow layer let us ship more in v4 than in all earlier versions together.
  • Conservative estimates build trust. Users prefer a plan that slightly under-promises to a training run that runs out of memory partway through.
  • Packaging is part of the product. If pip install doesn't give a working UI, nothing else matters.
  • Data normalization is most of fine-tuning. Strategies are the easy part. Getting messy real-world datasets into a consistent shape is where most of the work went.

What's next for ModelForge V4

  • Rewards for GRPO beyond exact match, including custom and model-graded rewards.
  • More use cases (classification and token tagging; the token-classification task already exists in the engine).
  • Measured memory profiling to tighten the conservative estimates.
  • More providers, such as Unsloth-style accelerated backends, on the new provider registry.
  • Evaluation dashboards that compare runs side by side.
  • Finishing the move into the main ForgeOpus/ModelForge repository, with full history, and publishing 4.0 on PyPI.

Built With

Share this project:

Updates

Submission history