Robust network pruning for enhanced accuracy under perturbations via structural and distributional consistency.

Abstract

Neural network pruning is a standard approach for reducing model size and inference cost. We revisit robustness-aware pruning and identify a strong correlation between intermediate feature stability and robustness degradation. This reveals that robustness is implicitly encoded in neural representations and can be exploited during pruning. Motivated by this, we propose a robustness-aware pruning method that selects channels based on their consistency under clean and perturbed inputs. Two complementary metrics are introduced: Structural-aware Consistency and Distributional-aware Consistency. To reconcile potential conflicts between them, we design a consistency-alignment fusion mechanism, SDA-Fuse, that jointly aligns and penalizes metric divergence, enabling stable and reliable neuron selection. Extensive experiments on corruption benchmarks (ImageNet-C, ImageNet-v2-C, ImageNet-C¯, and ImageNet-3DCC) demonstrate that our method achieves higher robustness with only a slight accuracy drop, achieving a better balance between clean accuracy and robustness.