SOURCE-LINKED INTELLIGENCE

Post-Training Science for Supervised Fine-Tuning

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Every supervised fine-tuning run forces the same chain of decisions, such as learning rate, batch size, LoRA or full fine-tuning, how many epochs, which optimiser, and what data to feed the model. Each of these is typically rediscovered from scratch for every new model and dataset. Here we measure them under one instrument: a sweep that varies one lever at a time, and spans dense and mixture-of-experts models in two families (Qwen3 and Llama), on four real-world customer SFT datasets, for both LoRA and full fine-tuning. These datasets give a controlled testbed: each task carries an evaluation

Read original source ↗ Open in workspace

recordType

paper

region

Global