SOURCE-LINKED INTELLIGENCE

Optimal Sequential Annotations for Off-Policy Evaluation

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Offline reinforcement learning and off-policy evaluation evaluates dynamic treatment rules based on retrospectively collected data prior to deployment. In recent AI applications, state and reward information is recorded as complex text or image, which recent AI advancements such as LLM-as-a-judge can label with unknown bias. Expert annotation may be available but at a higher cost. For example, safety classification via cheap but imperfect classifiers vs. expensive expert review. We show how a limited budget for ground-truth data-annotation can be used via doubly-robust OPE with missing rewards

Read original source ↗ Open in workspace

recordType

paper

region

Global