Boosting lightweight single image super-resolution via global prior feature.

Abstract

Recently, lightweight vision transformer (ViT)-based single image super-resolution (SISR) has gained significant attention. However, many existing lightweight methods struggle to achieve satisfactory performance due to the aggressive reduction in the number of parameters. Therefore, to improve the performance of lightweight networks, we propose a novel global feature prior self-attention network. First, conventional window-based self-attention methods typically apply attention mechanisms indiscriminately to all pixels within a window. This can lead to artifacts and texture blurring. To mitigate this issue, we leverage prior knowledge to identify texture-related pixels within the window and perform self-attention operations specifically on these pixels. Second, to enhance the network's ability to capture critical information and structural details, we introduce an efficient global feature extraction method. Finally, while transformers excel at capturing global features and low-frequency information, they often struggle with extracting local features and high-frequency information. Therefore, we integrate a local complementary module into the shift window attention to compensate for the transformer's shortcomings in extracting local and high-frequency features. Extensive experiments demonstrate that the proposed method outperforms all other state-of-the-art lightweight approaches. Code and models are obtainable at https://github.com/hms-source/GFPSAN.