AIIC AI Intelligence Centre

SOURCE-LINKED INTELLIGENCE

RM-VLM: From Natural-Language Task Descriptions to Vision-Grounded Reward Machines

arXiv · AI, language, vision and robotics · article · Nov 5, 2026 · UTC

Reward machines (RMs) have been proposed as a reward formulation in reinforcement learning (RL) to capture non-Markovian tasks where credit assignment depends on history. By representing reward functions as automata, RMs expose the task's temporal structure and allow such rewards to be handled within a Markovian framework. However, they assume a labelling function, a proposition-grounding oracle for observations, which must be manually engineered for each environment, limiting their wider applicability. In this work, we address this limitation by proposing RM-VLM, a novel approach in which a l

Read original source ↗ Open in workspace

recordType
paper
region
Global

Evidence & attribution

First collected: 2026-11-07T09:41:49.487Z. This is not the publication date.