SOURCE-LINKED INTELLIGENCE

Resource-Efficient Pruning for Transformer via Low-Rank Importance Estimation

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

With the rapid development of large-scale pre-trained language models based on Transformer architectures, their high computational and memory costs have become a major obstacle to deployment, especially in resource-constrained environments. Traditional pruning methods typically depend on full gradient-based importance estimation, and they necessitate prior finetuning of the model to achieve satisfactory performance. This process often results in intolerable resource consumption. This paper proposes REP-LIE, a new approach to enable resource-efficient pruning during the process of finetuning. R

Read original source ↗ Open in workspace

recordType

paper

region

Global