Kernelized linear principal component discriminant analysis.

Abstract

In this paper, we propose Kernelized Linear Principal Component Discriminant Analysis (KLPCDA), a structured and unified framework for discriminant analysis that overcomes the fragmentation in existing multi-stage approaches such as PCA+LDA or KPCA+GDA. Instead of treating feature extraction and class discrimination as disjoint steps, KLPCDA formulates a joint optimization model in the Reproducing Kernel Hilbert Space (RKHS), integrating overall variance preservation, between-class separation, and within-class compactness into a fused objective. The formulation supports seven KLPCDA variants, offering flexible control over effects of each criterion through tunable fusion coefficients. We present a systematic parameter optimization strategy, including kernel parameter selection, subspace dimensionality tuning, and fusion balancing, along with an alternative kernel parameter optimization method. Extensive experiments across image, tabular, and signal datasets across small-sample-size (SSS) to larger-scale settings validate the adaptability of KLPCDA. The results demonstrate KLPCDA consistently outperforms benchmark methods and convolutional neural networks in SSS settings on both recognition accuracy and efficiency, while maintaining competitive advantages in computational complexity and storage requirements for large-scale scenarios. Finally, we provide insights into extended study subjects and future work related to our proposal.