SOURCE-LINKED INTELLIGENCE

Importance-Aware Low-Rank Distillation of Diffusion Transformers

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Diffusion Transformers (DiTs) have emerged as a dominant architecture for high-quality text-to-image generation, yet their scale poses challenges for efficient deployment. While truncated singular value decomposition (SVD) is a principled tool for parameter reduction, evidence from large language models (LLMs) suggests that naive low-rank approximation can cause catastrophic failure. In contrast, we find that truncated SVD in DiTs produces smooth degradation even under substantial global compression, with redundancy distributed across projection matrices throughout the whole network rather tha

Read original source ↗ Open in workspace

recordType

paper

region

Global