SOURCE-LINKED INTELLIGENCE

All modalities are equal, but video is more equal: Closing the Cross-Attention Gap in Joint Video Generation

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Video is a rich representation of a physical event, capturing appearance, geometry, motion, and temporal evolution. Other modalities, such as 3D body motion or audio, encode narrower aspects of the same event. We find that joint multimodal diffusion transformers exhibit a corresponding asymmetry in cross-modal correspondence: companion modalities develop strong correspondences to video, but the reciprocal correspondences through which they constrain video remain substantially weaker. We express both directions as comparable correspondence distributions over video tokens and define their disagr

Read original source ↗ Open in workspace

recordType

paper

region

Global