NeuroDecoder: A new framework for image decoding and reconstruction of EEG signals.

Abstract

Brain-Computer Interface (BCI) technology holds great promise for enhancing human health and quality of life, with visual stimulus reconstruction from EEG signals being a key application. However, the complexity and noise of EEG data challenge existing reconstruction methods. To address these issues, we propose NeuroDecoder, an end-to-end multimodal guidance generation framework that produces high-quality images from EEG signals. The key innovation is the collaborative mitigation of EEG noise and cross-modal representation discrepancies through a noise-robust encoder, mask-based triple-contrastive alignment, and a fixed generative model. Specifically, NeuroDecoder consists of three integrated learning stages: 1) EEG Decoding, 2) Modality Alignment, and 3) Image Reconstruction. In the decoding stage, a novel visual decoding model extracts visually relevant features with superior classification accuracy. In the alignment stage, a mask-based triple contrastive learning strategy achieves efficient cross-modal alignment of EEG, text, image, and edge map embeddings into a unified space. In the generation stage, a new reconstruction pipeline feeds the aligned EEG embeddings into a pre-trained stable diffusion model, enabling high-quality visual stimulus reconstruction with enhanced semantic and structural fidelity, without fine-tuning the generative model. On three EEG datasets, NeuroDecoder achieved subject-dependent classification accuracies of 99.76%, 94.41%, and 56.67%, respectively; in the subject-independent setting, it performed near random on EEGCVPR40 but reached 91.61% and 37.63% on the other two. For image reconstruction, it obtained Fréchet Inception Distance of 62.84 and 63.12 on the first two datasets. Extensive experiments demonstrate that NeuroDecoder outperforms prior methods in both EEG classification accuracy and image reconstruction quality.