Pruning the ensemble of convolutional neural networks using second-order cone programming.

Abstract

Ensemble techniques are frequently encountered in machine learning and engineering problems since the method combines different models and produces an optimal predictive solution. The ensemble concept can be adapted to deep learning models to provide robustness and reliability. Due to the growth of the models in deep learning, using ensemble pruning is highly important to deal with computational complexity. Hence, this study proposes a mathematical model which prunes the ensemble of Convolutional Neural Networks (CNNs) consisting of different depths and layers that maximizes accuracy and diversity simultaneously with a sparse second order conic optimization model. The proposed model is tested on the CIFAR-10, CIFAR-100, and MNIST datasets, and its performance is compared with benchmark pruning methods, yielding promising results while reducing model complexity.