Decoding olfactory response from neurophysiological signal with a multi modal deep learning framework.

Abstract

The human olfactory system's temporal dynamics are crucial for sensory perception. By learning the temporal dynamics of EEG and utilizing breathing signals, we aim to better understand the neural features of olfactory perception from EEG. To decode the olfactory response effectively, we introduce a new method: the Token Alignment and Cross-Attention Fusion network (TACAF), a multimodal deep learning framework that enhances olfactory EEG decoding using wavelet features for time window selection and spectral analysis for data representation. Spatial features are extracted using spatial learning modules, and temporal dynamics are captured through a multi-head self-attention mechanism. The Temporal Token Semantic Alignment (TTSA) module synchronizes breathing information with EEG data for effective fusion. We collected EEG recordings and breathing signals from 20 subjects to study the decoding responses to pleasant and unpleasant odors. Our evaluation shows that TACAF significantly outperforms existing methods. Further analysis indicates that prolonged odor exposure leads to olfactory adaptation, reducing recognition performance. The findings are visualized through spatial topology maps with saliency mappings, providing insights into the neural mechanisms of olfactory perception.