SOURCE-LINKED INTELLIGENCE

Convergence Theory of Knowledge Distillation in Asynchronous P2P Gossip Learning Network

arXiv · AI, language, vision and robotics · article · Sep 1, 2026 · UTC

Decentralized, serverless learning increasingly connects devices running different architectures, where the standard tool, decentralized SGD, is undefined as models with different parameter counts cannot be averaged. Knowledge distillation (KD) exchanges soft predictions rather than weights and sidesteps this obstacle, yet convergence theory for fully decentralized, asynchronous peer-to-peer (P2P) KD is lacking. We provide one, relocating consensus from parameter space to function (output) space: a KD event is a geometric contraction operator in logit space on the peers' predictive distributio

Read original source ↗ Open in workspace

recordType

paper

region

Global