SOURCE-LINKED INTELLIGENCE

AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuning

arXiv · AI, language, vision and robotics · article · Aug 24, 2026 · UTC

Quantized fine-tuning (QLoRA) saves memory but not time. It dequantizes every 4-bit weight on the fly, so it trains more slowly than fp16 LoRA. We present AQLoRA (Adaptive-Quantization LoRA), a recipe that buys part of that time back. One CPU pass over the weights sets everything, with no search and no calibration data. The pass ranks layers by NF4 reconstruction error and keeps the top-K in fp16 under a memory budget. Those layers skip dequantization, which is where the speed comes from. A quality setting adapts every layer. A speed setting adapts only the top blocks, so the backward pass sto

Read original source ↗ Open in workspace

recordType

paper

region

Global