SOURCE-LINKED INTELLIGENCE

SpikeOPD: Stable On-Policy Distillation for Autoregressive Spiking Language Models

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Spiking neural networks (SNNs) offer a path to energy-efficient language modeling through sparse encoding and event-driven computation, but training capable spiking language models from scratch remains difficult. A practical alternative is ANN-to-SNN migration through knowledge distillation (KD), where a pretrained artificial neural network (ANN) teacher supervises an SNN student. Existing migration approaches distill on fixed corpus prefixes, whereas autoregressive inference conditions on self-generated prefixes, creating prefix-source mismatch. It manifests as output-policy mismatch with the

Read original source ↗ Open in workspace

recordType

paper

region

Global