EEGMamba: An EEG foundation model with Mamba.

Abstract

Electroencephalography (EEG) captures brain activity and has been widely used in clinic and brain-computer interfaces (BCIs). Classic EEG decoding methods rely on supervised learning, limiting their performance and generalizability. Inspired by the revolutionary impact of large models in other fields, researchers are now investigating EEG foundation models. Recently, state space models (SSMs), such as Mamba, have demonstrated strong sequence modeling capabilities, which may be suitable to model the spatiotemporal dependencies of EEG signals. However, the application of Mamba for EEG representation learning remains largely unexplored. In this paper, we investigate the potential of Mamba for learning generic EEG representations and propose a novel EEG foundation model, EEGMamba. Specifically, we employ Mamba encoder as the backbone of EEGMamba to model the spatiotemporal dependencies among EEG patches. Meanwhile, we use patch-based masked EEG reconstruction to learn generic EEG representations. EEGMamba is pre-trained on a large and diverse EEG corpus (16,724 h) from five datasets. We evaluate EEGMamba on up to six downstream BCI tasks using six public datasets. EEGMamba achieves the state-of-the-art performance across all the tasks, demonstrating its strong capability and generalizability.