A multi-level teacher assistant-based knowledge distillation framework with dynamic feedback for motor imagery EEG decoding.

Abstract

Deep learning has shown promise in motor imagery-based electroencephalogram (MI-EEG) decoding, a critical task in non-invasive brain-computer interfaces (BCIs). In response to the computational complexity of deep learning models to be deployed in practical BCI applications, knowledge distillation (KD) has emerged as a solution for model compression. However, vanilla KD methods struggle to effectively extract and transfer the abundant multi-level knowledge from MI-EEG signals under high compression ratios. This study proposes a novel knowledge distillation framework termed Motor Imagery Knowledge Distillation (MIKD), which compresses deep learning models for MI classification tasks while maintaining high performance. The MIKD framework consists of two key modules: (1) a multi-level teacher assistant knowledge distillation (ML-TAKD) module designed to extract and transfer local representations and global dependencies of MI-EEG signals from the complex teacher network to the much smaller student network, and (2) a dynamic feedback module that allows the teacher assistant to adjust its teaching strategy based on the student's learning progress. Extensive experiments on three public EEG datasets demonstrate that the MIKD framework achieves state-of-the-art performance. The proposed framework improves the baseline student model's accuracy by 6.61 %, 1.91 %, and 3.29 % on the three datasets, while reducing the model size by nearly 90 %.