SOURCE-LINKED INTELLIGENCE

Detecting Pretraining Data in Large Language Models from a Free-Energy Perspective

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Detecting pretraining data in large language models is challenging because high likelihood can reflect either training exposure or strong generalization. In the joint space of prediction loss and predictive entropy, a likelihood-only detector uses a horizontal boundary and can mistake predictable non-members for members. Motivated by this, we introduce an inclined boundary that evaluates prediction loss relative to predictive entropy. Our analysis shows that entropy correction can preserve the expected membership signal while reducing its variance, thereby improving standardized member--non-me

Read original source ↗ Open in workspace

recordType

paper

region

Global