SOURCE-LINKED INTELLIGENCE

How Does mHC Use Its Residual Streams? Selective Routing and Near-Identity Mixing

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Hyper-Connections and their manifold-constrained variant mHC widen a residual pathway from one stream to n, yet how trained models use this capacity remains unclear: how broadly blocks read and write, how strongly the residual pathway mixes streams, and whether the streams carry distinct representations. We examine these properties in the four-stream residual pathway of DeepSeek-V4-Flash using effective stream counts, cross-stream residual weights, and inter-stream cosine similarity. Read/write routing is concentrated but varies across depth: a typical attention or FFN site effectively uses ab

Read original source ↗ Open in workspace

recordType

paper

region

Global