Learned low-rank representation and its theoretical convergence analysis.

Abstract

The extraction of patterns and knowledge from high-dimensional data is very important yet challenging. To uncover their underlying structures, sparse representation and low-rank approximation have emerged as fundamental tools for leveraging prior knowledge of data structures. Recent advances in sparse and low-rank modeling, particularly through algorithm unfolding into deep neural networks, have further led to remarkable improvements in performances. Although there has been a lot of work on theoretical investigations and practical applications of algorithm unrolling models for sparse representation, the theoretical framework for algorithm unfolding based on low-rank representation remains largely unexplored, with few convergence guarantees and limited studies. To address these challenges, we first propose a novel unfolded deep network for low-rank representation, termed Learned Low-Rank Representation (LLRR), and further introduce an enhanced variant with a partial weight coupling mechanism, referred to as LLRR with Partial Weight Coupling (LLRR-PWC). Subsequently, we conduct an in-depth theoretical analysis of the convergence properties of the LLRR-PWC model by innovatively designing an appropriate network parameter space. On this basis, we not only rigorously ensure the convergence of the low-rank unfolded network architecture, but also achieve a significant improvement in the convergence rate both theoretically and empirically. Finally, to validate our theoretical claims and the practical advantages of our LLRR-PWC, we conduct a comprehensive series of experiments, demonstrating our theoretical findings and highlighting the practical value and applicability of our LLRR-PWC algorithm.