HSA²M: Hierarchical Spike Aggregation Activation Map for Visual Explanations From Spiking Neural Networks.

Abstract

Achieving visual explanations in spiking neural networks (SNNs) is challenging due to the spatial sparsity and temporal discontinuity of spikes, which make it difficult to generate coherent saliency mappings and hinder interpretability. To address this, we propose hierarchical spike aggregation activation map (HSA²M), a method inspired by two neurobiological principles: 1) hierarchical feature integration in the ventral pathway (V $1\rightarrow $ V $2\rightarrow $ V $4\rightarrow $ IT) and 2) short interspike intervals (ISI) reflecting decision-relevant saliency. HSA²M achieves fine-grained visual explanations in SNNs through multilayer spike aggregation and adaptive fusion. It consists of three core modules: a spike activation map generator (SAMG) that constructs layer-wise saliency maps via temporal spike aggregation, a Fisher-weighted fusion (FWF) module that adaptively integrates multilayer maps using Fisher information (FI), and a metric-aware hyperparameter optimizer (MA-HPO) that enhances explanatory fidelity through metric-driven parameter tuning. By integrating these components, HSA²M generates more precise and high-fidelity visual explanations while maintaining event-driven efficiency through spike-based processing. Extensive experiments on neuromorphic (DVS-Gesture and DSEC-Semantic) and static (Tiny-ImageNet and ImageNet) benchmarks demonstrate that HSA²M outperforms state-of-the-art methods in terms of interpretability [e.g., ADCC improved from 0.8792 to 0.9215 (+4.81%)], faithfulness [e.g., Spearman's rank correlation coefficient $\rho $ improved from 0.189 to 0.209(+10.085%)], and adversarial robustness [e.g., normalized L1 distance decreased from 0.0673 to 0.0092 (-86.32%)].