SOURCE-LINKED INTELLIGENCE

PSD: Pseudo Self-Distillation of Memory Representation Capabilities for LLM Agents

arXiv · AI, language, vision and robotics · article · Sep 20, 2026 · UTC

Memory systems are becoming a core component of LLM agents, but constructing and maintaining memory remains expensive because it relies on repeated calls to large proprietary language models. This cost creates a major barrier to deploying memory-enhanced agents at scale. In this paper, we present Pseudo Self-Distillation (PSD), a framework that enables small language models (SLMs) to construct hierarchical memory representations by distilling behavior from a strong black-box oracle through a multi-stage training pipeline. Standard distillation methods require access to teacher logits or hidden

Read original source ↗ Open in workspace

recordType

paper

region

Global