SOURCE-LINKED INTELLIGENCE

Behaviorally Effective LoRA Writes Are Sparse and Structured

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Low-rank adaptation fixes the rank of the update, but it does not identify which parts of a trained write actually carry behavior. We study that question directly and show that behaviorally effective LoRA writes are sparse, structured, and far more concentrated than the raw low-rank parameterization suggests. We use Learned-Basis LoRA, a learned-basis continuation recipe, to expose that structure. The recipe warms up an unconstrained adapter, converts its learned write columns into a module-wise orthonormal basis, freezes that basis, and continues training inside the constrained parameterizati

Read original source ↗ Open in workspace

recordType

paper

region

Global