FoxMind 49M is a compact language model with 49,430,016 parameters, built to explore how much reasoning, accuracy, and instruction-following ability can be achieved under a strict 50M-parameter limit.
Instead of relying on a single long training run, I built an iterative training pipeline that evaluates the model throughout training, saves checkpoints, adapts training priorities, and keeps an earlier checkpoint when later training makes the model worse.
The project focuses on efficient AI development using consumer hardware. FoxMind was trained locally on an AMD Radeon RX 7900 XTX and uses a mix of language modeling, multiple-choice training, ranking, math reasoning, arithmetic, instruction following, and general-purpose data.
One of the biggest challenges was balancing capabilities. Training heavily on one skill could improve that metric while hurting another. Because of that, the system uses held-out evaluation and anti-regression checks to decide which checkpoint should become the final model.
The current best model comes from the V16 training run and remains under 50 million parameters while achieving strong instruction-following and improved benchmark performance compared with earlier versions.
