Genetic Algorithm-Based Two-Step Optimization for Precise Latent Factor Analysis.

Abstract

The latent factor analysis (LFA) model is an effective tool for extracting valuable information from high-dimensional and sparse (HiDS) matrices. However, traditional LFA usually suffers from low accuracy due to the limitations of the stochastic gradient descent (SGD) algorithm used in its model training. First, the learning rate of SGD is adjusted manually, which greatly affects the training efficiency. A recent solution is adjusting this hyperparameter by the particle swarm optimization (PSO) algorithm. However, PSO cannot adapt well to the dynamic decision space of this problem due to its strong convergence. Second, SGD relies solely on the gradient information to perform the optimization, which decreases the training accuracy. To address the above two issues, this article proposes a novel LFA model called genetic algorithm-based two-step LFA (GA-TSLFA), which employs the GA to facilitate the model training. Compared to PSO, the GA has better flexibility, which can be employed to tune the hyperparameter of LFA in dynamic decision spaces and refine the model in high-dimensional and complex decision spaces by designing suitable evolutionary operators. The training of the proposed GA-TSLFA consists of two steps. In the first step, the model is pretrained by SGD whose learning rate is adaptively adjusted by a proposed GA. In the second step, the LF matrices generated by SGD are further refined using a proposed GA-based framework. This framework operates by optimizing a subset of partial vectors, which are selected through a dedicated strategy. In this way, the model's accuracy can be further enhanced. Empirical studies on benchmark datasets show that the GA-TSLFA surpasses state-of-the-art LFA models in prediction accuracy and has a competitive efficiency.