Self-Expressive High-Order Tensor Unrolling Network for Unsupervised Hyperspectral and Multispectral Image Fusion.

Abstract

Hyperspectral and multispectral image fusion (HMF) enhances spatial-spectral quality by fusing low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI). Although recent fusion methods have shown promise in preserving the multi-mode structure of high-dimensional data, existing fusion methods still face some challenges. For tensor-based approaches, conventional mode-wise decomposition, such as order-3 CP or Tucker decomposition, may disrupt intrinsic spatial consistency. Furthermore, although deep learning exhibits powerful feature representation ability, existing deep fusion methods either rely on 'black-boxed' deep fusion networks remain insufficiently interpretability with large training data. To address these issues, a novel Self-Expressive High-Order Tensor Unrolling Network (SHOTUN) is proposed for unsupervised HSI-MSI fusion. Within the sparse core tensor decomposition framework, we introduce the intrinsic self-expressive relationships among overlapping image patches as a form of high-order mode representation to preserve spatial structure of the fusion model. During optimization, we adopt an alternative optimizing strategy and design dedicated modules for each sub-problem, yielding an interpretable end-to-end training pipeline. Furthermore, to improve generalization across different sensors, we introduce a pre-training strategy into the unsupervised training for the more accurate estimation of unknown degraded parameters. Extensive experimental results on simulated and real datasets demonstrate the effectiveness of our proposed method. The source code will be publicly available at https://github.com/Shawn-H-Wang/SHOTUN.