SOURCE-LINKED INTELLIGENCE

Confess What You Know: Forget-Set Misalignment with Model Knowledge in LLM Unlearning

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Machine unlearning for large language models (LLMs) often assumes that a pre-defined forget set matches what the model has memorized, but this frequently breaks in realistic privacy settings where the original training data is inaccessible. We term this gap forget-set misalignment and identify two cases. In Under Unlearning, the forget set omits memorized information and leakage persists. In Out-of-Knowledge Unlearning, the algorithm is driven to "forget" knowledge the model never learned, perturbing parameters and degrading utility. Using gradient-level analysis, we show these behaviors arise

Read original source ↗ Open in workspace

recordType

paper

region

Global