Adaptive fast local discriminant analysis with whitening transform.

Abstract

Linear discriminant analysis (LDA) is a dimensionality reduction method appreciated by many researchers in the field of machine learning. LDA is usually summarized as a trace ratio problem: min_WTr(W^TS_wW)/Tr(W^TS_tW), which has clear physical meaning but is difficult to solve. Generally, in order to facilitate the solution, researchers often solve the following ratio trace problem: min_WTr((W^TS_wW)/(W^TS_tW)) instead of directly solving the above trace ratio problem. The ratio trace problem can be solved in a closed form, but what is its physical meaning? In addition, most existing LDA-related works consider the relationships between all data points to explore the local structure of the data, and these methods are computationally expensive and difficult to be widely used in large-scale data scenarios. In this work, we illustrate the physical meaning of the ratio trace: whitening the data and then making its intra-class distance as small as possible. Furthermore, we propose an adaptive fast local discriminant analysis method with whitening transformation (FLDA-W), in which the degree of similarity between sample points and anchors is explored in the optimal subspace with the help of the anchor strategy, and in order to ensure the k-connectivity of the similarity matrix, we impose a ℓ₀-norm constraint on it. Finally, experiments conducted on the three-ring datasets and nine real-world datasets illustrate that FLDA-W is robust to noise and also has excellent performance in classification accuracy.