A robust outlier control framework for classification designed with family of homotopy loss function.

Abstract

We propose a new homotopy loss, where practitioners can tune the parameter to derive different loss functions such as l₁-norm loss, logarithmic loss, Geman-Reynolds loss, Geman-McClure loss and correntropy-based loss. Moreover, we illustrate that the proposed loss satisfies Fisher consistency, and we analyze the robustness of the proposed homotopy loss from different perspectives: M-estimation and adversarial perturbations. Then, we represent a new evaluation standard to measure robustness and demonstrate its upper bound to ensure the validity of this measure. Applied the proposed homotopy loss to least square support vector machine (LSSVM) and the extreme learning machine (ELM), two robust models are presented to enhance the robustness. Furthermore, re-weighted least square algorithm is used to solve the problems, and the resulting algorithms converge globally. In addition, the proposed methods are implemented on various datasets with different levels of noise. Compared with traditional methods, experiment results on real-world datasets show that the proposed methods have superior anti-interference ability to outliers in most cases.