CNNViT-MILF-a: A Novel Architecture Leveraging the Synergy of CNN and ViT for Motor Imagery Classification.

Abstract

Accurate motor imagery (MI) classification in EEG-based brain-computer interfaces (BCIs) is essential for applications in engineering, medicine, and artificial intelligence. Due to the limitations of single-model approaches, hybrid model architectures have emerged as a promising direction. In particular, convolutional neural networks (CNNs) and vision transformers (ViTs) demonstrate strong complementary capabilities, leading to enhanced performance. This study proposes a series of novel models, termed as CNNViT-MI, to explore the synergy of CNNs and ViTs for MI classification. Specifically, five fusion strategies were defined: parallel integration, sequential integration, hierarchical integration, early fusion, and late fusion. Based on these strategies, eight candidate models were developed. Experiments were conducted on four datasets: BCI competition IV dataset 2a, BCI competition IV dataset 2b, high gamma dataset, and a self-collected MI-GS dataset. The results demonstrate that CNNViT-MILF-a achieves the best performance among all candidates by leveraging ViT as the backbone for global feature extraction and incorporating CNN-based local representations through a late fusion strategy. Compared to the best-performing state-of-the-art (SOTA) methods, mean accuracy was improved by 2.27%, 2.31%, 0.74%, and 2.50% on the respective datasets, confirming the model's effectiveness and broad applicability, other metrics showed similar improvements. In addition, significance analysis, ablation studies, and visualization analysis were conducted, and corresponding clinical integration and rehabilitation protocols were developed to support practical use in healthcare.