SOURCE-LINKED INTELLIGENCE

An Action Is Worth One Patch: Unified World-Action Modeling with PatchWAM

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Generative visual models offer a foundation for learning representations of physical dynamics, yet their extension to continuous control raises a fundamental question: do visual prediction and action generation require separate computational pathways? Existing approaches usually introduce trainable action heads or separate action experts to bridge low-dimensional states and high-dimensional visual representations. In this work, we explore whether the visual backbone's existing capacity can also support control when actions are expressed in a compatible representation. Thus, we introduce PatchW

Read original source ↗ Open in workspace

recordType

paper

region

Global