Sparse Bayesian Broad Learning System via Adaptive Lasso Priors for Robust Regression.

Abstract

Broad learning system (BLS), as an innovative type of neural network, has demonstrated exceptional performance in regression tasks. Nonetheless, the majority of BLS methods, which rely on the least squares criterion, are highly sensitive to outliers and noisy data, resulting in reduced prediction accuracy. To improve the robustness of broad networks, a sparse Bayesian BLS via adaptive Lasso priors (AL-SBBLS) is proposed in this article to handle regression tasks with data contaminated by outliers and noise. Specifically, adaptive Lasso constraints are first applied to enhance the adaptive sparsity of output weights, which facilitates the automatic selection of highly correlated features. Subsequently, a multilayer Bayesian framework is constructed to provide an adaptive Lasso prior to the output weights, allowing the model for the adaptive learning of regularization factors and the estimation of probability distributions for output values, while further sparsifying the network. By selecting highly correlated features and estimating the probability distributions of output values, the impact of outliers and noise can be effectively mitigated. To effectively train the networks, corresponding optimization algorithms are designed for AL-SBLS and AL-SBBLS using the alternating direction method of multipliers (ADMMs) and variational Bayesian inference methods, respectively. The effectiveness and robustness of the proposed methods are validated through robust regression experiments on 14 real-world datasets and complex nonlinear data. Quantitative results demonstrate that the proposed AL-SBBLS achieves the best performance on most datasets, attaining the lowest average ranking of 1.44 in Friedman tests compared with 11 state-of-the-art BLS variants, which confirms its superior predictive accuracy and robustness. The resource code of AL-SBBLS proposed in this article is available at: https://github.com/taocheny/AL-SBBLS.