SOURCE-LINKED INTELLIGENCE

Let the Neurons Die: Exploiting ReLU-Induced Model Degradation

arXiv · AI, language, vision and robotics · article · Sep 28, 2026 · UTC

Rectified linear unit (ReLU) networks can suffer from dying neurons, where units with persistently negative pre-activations produce zero outputs, blocking gradients through their activations. To exploit this failure mode, we present three training-time availability attacks based on data ordering and poisoning. We begin with the basic dynamic data-ordering attack (DOA), which greedily constructs a training prefix by selecting the next example that minimizes the target layer's post-update weight sum, aiming to push ReLU units toward negative pre-activations without modifying training samples or

Read original source ↗ Open in workspace

recordType

paper

region

Global