SOURCE-LINKED INTELLIGENCE

Repurposing Pre-trained LLMs as High Fidelity Continuous Text Autoencoders

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

Next-token prediction has enabled highly fluent autoregressive language models, but it represents global structure only indirectly through sequential factorization. In contrast, high-fidelity autoencoders have become a standard primitive in image generation, enabling generative models to operate over continuous latent spaces; text lacks a comparably faithful continuous representation. We propose LLMAE, a method for repurposing a pretrained decoder-only language model as a continuous text autoencoder by exposing an intermediate fixed-length latent bottleneck within its internal activations. Ins

Read original source ↗ Open in workspace

recordType

paper

region

Global