Efficient High-Dimensional Learning With Adaptive Gaussian RBF Networks.

Abstract

Radial basis function neural networks (RBFNNs) are widely applied due to their rapid modeling capabilities and efficient learning performance. However, when dealing with high-dimensional data, RBFNNs encounter two critical limitations: the hidden layer responses using Gaussian kernels suffer from ineffective activation and numeric underflow; and the estimation of output layer weights typically involves tedious parameter tuning and inefficient loading of high-dimensional feature matrices. To overcome these challenges, we first propose a dimensionality-adaptive Gaussian kernel function (DAGKF) equipped with a novel width adjustment mechanism that flexibly mitigates the numerical difficulties inherent in high-dimensional spaces. Moreover, to avoid processing entire feature matrices simultaneously, we introduce a multioutput coordinate descent (MOCD) algorithm that enables parallel computation across multioutput systems. Building upon MOCD, we further develop the joint residual MOCD (JRMOCD) algorithm, which incorporates a joint residual criterion for more effective weight estimation. The convergence of the JRMOCD algorithm is rigorously proven. Extensive experiments demonstrate the superior performance of the proposed methods, particularly in high-dimensional settings.