Location Property of Convolutional Neural Networks for Image Classification.
basic_science · Level V
Where this comes from
- Record sourced from PubMed, PMID 32841123.
- Also identified by DOI 10.1109/TNNLS.2020.3015965.
- No licence information is recorded for this record.
- Because redistribution is not established, this page shows the abstract only. Follow the links below for the full text.
Abstract
When doing image classification, the core task of convolutional neural network (CNN)-based methods is to learn better feature representation. Our analysis has shown that a better feature representation in the layer before softmax operation (BSM-layer) means a better feature embedding location that has a larger distance to the separating hyperplane. By defining this property "Location Property" of CNN, the core task of CNN-based methods can be regarded as to find out the optimal feature embedding location in the BSM-layer. In order to achieve this, in this work, we first propose two feature embedding directions, principal embedding direction (PE-direction) and secondary embedding direction (SE-direction). And then, we further propose a loss-based optimization framework, location property loss (LP-loss), which can make feature representation move in the PE-direction and the SE-direction simultaneously during the training phase. LP-loss consists of two parts, LP<sub>PE</sub> and LP<sub>SE</sub>, where LP<sub>PE</sub> focuses on PE-direction, and LP<sub>SE</sub> focuses on SE-direction. Any loss function focusing on these two embedding directions can be chosen as LP<sub>PE</sub> and LP<sub>SE</sub>. Based on the analysis that softmax, L-softmax, and AM softmax can make the feature representation move in PE-direction to a different extent, any of them can be chosen as LP<sub>PE</sub>. Since there is no existing works can fulfill the purpose of LP<sub>SE</sub>, a novel loss, secondary optimal feature plane loss (S-OFP loss), is developed. S-OFP loss is designed to make feature representations belonging to the same category embed onto their corresponding S-OFP. It is proved that S-OFP loss is the optimal feature plane in the SE-direction. Experiments are done with shallow, moderate, and deep models on four benchmark data sets, including the MNIST, SVHN, CIFAR-10, and CIFAR-100, and results demonstrate that CNN models can obtain remarkable performance improvements with LP<sub>softmax</sub>, S-OFP and LP<sub>AM softmax</sub>, S-OFP, which verify the effectiveness of location property.