SOURCE-LINKED INTELLIGENCE

Can Video World Models Track Unobserved World States?

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Video world models are increasingly used as simulators, yet visual fidelity alone does not show that a model maintains the hidden state of the world. We examine this gap with an action-conditioned video Shell Game, a visual analog of $S_5$ state tracking that decouples visual rendering from compositing the hidden state underneath. Bidirectional and autoregressive Transformers, Mamba, and linear attention restricted to nonnegative transition eigenvalues all fit the training horizon of 5 swaps and then fall toward chance on longer swap chains (extrapolation) while still rendering plausible video

Read original source ↗ Open in workspace

recordType

paper

region

Global