AIIC AI Intelligence Centre

SOURCE-LINKED INTELLIGENCE

LACCO: Local Accumulation with Computation--Communication Overlap

arXiv · AI, language, vision and robotics · article · Nov 13, 2026 · UTC

Training language models across low-bandwidth networks requires reducing synchronization overhead. Local optimization methods such as DiLoCo communicate only after several local steps, but outer gradient averaging and the outer update remain on the critical path. Existing computation-communication overlap methods avoid this stall through delayed or replica-local updates, which can degrade convergence or cause replicas to drift. We propose LACCO (Local Accumulation with Computation-Communication Overlap), a two-stage algorithm that overlaps global communication with local computation while appl

Read original source ↗ Open in workspace

recordType
paper
region
Global

Evidence & attribution

First collected: 2026-11-16T10:41:26.071Z. This is not the publication date.