SOURCE-LINKED INTELLIGENCE

Denoising Diffusion Generative Models Secretly Calculate Attentions

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Denoising diffusion models are the dominant architecture for image generation, whereas most natural language generation and modeling are primarily handled by well-known transformer architectures employing attention mechanism. Here, we show that diffusion models also inherently use an attention mechanism very similar to that of transformers. Therefore, attention emerges as a universal machine learning principle, based on a general training objective. We also show similarities in basic functional principle of auto-encoders and attention-based models. These equivalences allows us to interchange t

Read original source ↗ Open in workspace

recordType

paper

region

Global