Multimodal Spiking Neural Network With Generalized Distributive Law for Biosignal and Sensory Fusion.

Abstract

Multimodal signal fusion is a cornerstone of biomedical engineering and intelligent sensing, enabling holistic analysis of heterogeneous sources such as electroencephalography (EEG), peripheral signals, speech, and imaging data. However, integrating diverse modalities in a computationally efficient and biologically plausible manner remains a significant challenge. Transformer-based fusion architectures rely on global cross-attention to integrate multimodal information but incur high computational costs. In contrast, STDP-driven fully connected layers adopt local learning rules, which restrict their ability to autonomously form efficient sparse topologies for complex multimodal tasks. To address these issues, we propose a novel end-to-end framework-the Multimodal Spiking Neural Network (MSNN)-featuring a fusion module grounded in the Generalized Distributive Law (GDL). This principled mechanism provides an efficient and interpretable means of integrating heterogeneous biomedical and sensory signals. The MSNN further incorporates structure-adaptive leaky integrate-and-fire (SALIF) neurons, enabling dynamic optimization of sparse connectivity to enhance fusion efficiency. The proposed MSNN is validated on a range of datasets, demonstrating strong versatility: it achieves binary classification accuracies of 92.29% (valence) and 91.08% (arousal) on the DEAP dataset for affective state decoding and 99.77% on the WESAD dataset for stress detection, while delivering state-of-the-art performance on standard pattern recognition tasks (MNIST & TIDIGITS: 99.01%) and event-driven neuromorphic datasets (MNIST-DVS & N-TIDIGITS: 99.98%). These results demonstrate that MSNN offers an effective and energy-efficient solution for multimodal sensor fusion in biomedical and intelligent sensing applications.