Butterfly Residual Network: A Hybrid Approach With Spectral Transformers and Depth-Wise Convolutions for Hyperspectral Image Super-Resolution.

Abstract

Hyperspectral image (HSI) super-resolution reconstruction is a challenging ill-posed inverse problem, which seeks to enhance the spatial resolution of low-resolution hyperspectral images (LR-HSIs) by integrating complementary information from high-resolution multispectral images (HR-MSIs), ultimately generating high-resolution HSIs (HR-HSIs). Existing methods commonly employ residual connections and deep layer stacking to facilitate information propagation. While residual connections effectively preserve gradient flow, we observe that naively increasing network depth in high-dimensional spectral tasks can lead to feature redundancy and performance saturation. To address these challenges, this article presents a novel Butterfly residual network (BRNet) that incorporates spectral Transformers and depth-wise convolutions to optimize both accuracy and computational efficiency of hyperspectral super-resolution reconstruction from two perspectives: learning strategy and feature extraction. Regarding learning strategy, a recursive structure coupled with a fusion parameter generation technique is proposed to promote efficient feature fusion and enable adaptive network pruning, thereby reducing redundant information and enhancing computational efficiency. For feature extraction, spectral Transformer and depth-wise convolution are employed to capture spectral and spatial features, respectively, effectively leveraging their complementary advantages across different dimensions. A specialized spectral-spatial interaction (SSI) module is then incorporated to effectively fuse the extracted features, thereby enriching the diversity of network features. Additionally, the convolutional gated feed-forward network (FFN) is designed to bolster the network's ability to capture local features while significantly reducing the computational complexity. Experimental evaluations on three hyperspectral datasets demonstrate that the proposed method outperforms existing state-of-the-art super-resolution reconstruction methods across various performance metrics, validating its effectiveness and superiority.