SOURCE-LINKED INTELLIGENCE

Noisy-Space Policy Gradient for Diffusion Policies in Offline Reinforcement Learning

arXiv · AI, language, vision and robotics · article · Sep 7, 2026 · UTC

Diffusion policies offer a powerful and expressive parameterization for continuous control. Yet, their integration with reinforcement learning remains conceptually and algorithmically challenging. In this work, we address this gap by introducing a noisy-space action-value (Q-)function that assigns values to diffusion latents through the distribution of executed actions induced by the denoising process. We show that this construction admits a precise semantic interpretation and derive a noisy-space policy gradient (NSPG) that optimizes noisy latents using only clean action-space value estimates

Read original source ↗ Open in workspace

recordType

paper

region

Global