Distributed Min-Max Learning Scheme for Neural Networks With Applications to High-Dimensional Classification.

Abstract

In this article, a novel learning methodology is introduced for the problem of classification in the context of high-dimensional data. In particular, the challenges introduced by high-dimensional data sets are addressed by formulating a L₁ regularized zero-sum game where optimal sparsity is estimated through a two-player game between the penalty coefficients/sparsity parameters and the deep neural network weights. In order to solve this game, a distributed learning methodology is proposed where additional variables are utilized to derive layerwise cost functions. Finally, an alternating minimization approach developed to solve the problem where the Nash solution provides optimal sparsity and compensation through the classifier. The proposed learning approach is implemented in a parallel and distributed environment through a novel computational algorithm. The efficiency of the approach is demonstrated both theoretically and empirically with nine data sets.