SOURCE-LINKED INTELLIGENCE

Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks

arXiv · AI, language, vision and robotics · article · Sep 3, 2026 · UTC

Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging methods assume every layer needs the same rank budget. Further, some methods assume that rank budget needs to be split equally among the tasks too. We show this uniform-budget assumption is a major source of the performance gap between merged and per-task LoRAs. However, rank selection is an NP hard problem. To this end, we introduce Net Utility, a data free metric that first decomposes every task LoRA by its Singular Value Decomposition (SVD) and

Read original source ↗ Open in workspace

recordType

paper

region

Global