SOURCE-LINKED INTELLIGENCE

Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Marginalized importance weighting evaluates a target policy by reweighting offline state-action samples with its discounted occupancy ratio, characterized by an adjoint Bellman equation. Existing minimax, primal-dual, and fitted fixed-point estimators can leave residual occupancy-balance violations because of function-class approximation, regularization, or incomplete optimization. These violations are difficult to diagnose and reduce because the objectives generally lack a direct supervised validation loss for hyperparameter tuning, model selection, and early stopping. We introduce isotonic B

Read original source ↗ Open in workspace

recordType

paper

region

Global