Tiny Data Is Sufficient: A Generalizable CNN Architecture for Temporal Domain Long Sequence Identification.

Abstract

Deep learning (DL) models have made remarkable progress in various sequence processing tasks. It is widely acknowledged that these models heavily rely on numerous training data and finely tuned parameters. Recent studies highlighted that conventional convolutions in deep networks may hamper feature processing efficacy, particularly in long temporal sequence analysis, due to their limited feature representation capabilities. To tackle these challenges, this article introduces a novel generalizable convolutional neural network (GeCNN) architecture tailored for temporal domain long sequence identification. Our framework incorporates three key components: the generic convolutional neural network (CNN), selective CNN, and multiple pooling layers. The generic CNN implements customizable hyper-convolutional operations through non-linear convolvers, thereby enhancing feature representation effectiveness and significantly improving accuracy. Subsequently, the selective CNN is designed to abate the demand for large training data by focusing on various subsequences. We propose the homogeneous striding principle and the partial homogeneous striding theorem to theoretically support the method. The multiple pooling combines eight distinct pooling operations to mitigate the statistical information loss problem typically associated with single pooling actions. Experimental results demonstrate that our GeCNN architecture achieves superior performance with shallow networks and tiny data compared to existing deep networks. The accuracy of the best-trained model surpasses the ResNet and self-attention-based models by 9.51% and 16.79% utilizing only 0.18% of data for training in the GTZAN dataset. Meanwhile, the accuracy of the optimal model overtakes the other two models by 5.35% and 10.16% while using merely 1.56% of data for training in the PLAID dataset.