SOURCE-LINKED INTELLIGENCE

Lifted Bellman Linear Programming for Offline Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 21, 2026 · UTC

Offline reinforcement learning (RL) typically trains a critic by minimizing a regression loss against bootstrapped value targets stabilized by target networks with exponential moving average (EMA) updates. Multi-step targets incorporate behavior-policy actions and therefore require off-policy correction. We instead impose in-sample Bellman optimality on the critic through inequality constraints. We formulate the Lifted Bellman Linear Program (LBLP), which lifts the linear programming characterization of Bellman optimality to the joint $(Q,V)$ space so that every constraint involves only state-

Read original source ↗ Open in workspace

recordType

paper

region

Global