CS-Net: convolutional spider neural network for surface-EMG-based hybrid gesture recognition.

Abstract

<i>Objective.</i>In this paper, we propose a novel neural network architecture, the convolutional spider neural network (CS-Net), combined with a transfer learning (TL) strategy, to classify hybrid gestures that integrate wrist postures and hand movements.<i>Approach.</i>The CS-Net framework incorporates diverse surface electromyography (sEMG) features, including raw signals and FFT representations, through a multi-stream information fusion mechanism to enhance classification performance. The proposed TL strategy involves pre-training the model on specific wrist postures and fine-tuning it on the full set of hybrid gestures, leveraging the intrinsic relationships between composite gestures and their constituent movements to improve accuracy. The framework is evaluated through extensive offline experiments using a dataset of 12 hybrid gestures (combining three wrist postures and four hand movements) collected from six subjects, comparing its performance against three deep learning algorithms in sEMG recognition filed.<i>Main results.</i>The average experimental result for the proposed CS-Net with TL reached 90.6%. Additionally, its generalization ability is validated with the Ninapro public databases, which are DB1, DB4, and DB5. The 30 action classification accuracy of CS-Net on the Ninapro datasets was 68.7%, 61.5% and 66.3%, respectively. To demonstrate practical applicability, real-time online experiments involving object grasping tasks is conducted, achieving a success rate of 90%.<i>Significance.</i>The results show that CS-Net significantly improves sEMG classification accuracy, while the TL strategy further enhances performance. Moreover, the algorithm achieved a high success rate in online experiments, confirming its robustness and practical utility for real-world applications. Our hybrid gesture dataset and source codes are available on Github.