Spectral Manifold Variational Network for Hyperspectral Superresolution With Spectral Variability.

Abstract

Given the inherent tradeoff between spectral and spatial resolution, hyperspectral images (HSIs) typically exhibit insufficient spatial details. Combining the HSI with a corresponding high spatial resolution conventional imagery serves as a compromised alternative to produce a high-quality HSI. While fusion-based HSI superresolution (HSR) schemes have demonstrated effectiveness in this task, the spectral variability arising from the fusion of heterogeneous images remains an underexplored challenge in prior research. To this end, this article presents a novel zero-shot fusion model for HSR accounting for spectral variability between fusion materials, where only the test-time inputs are used to learn an image-specific fusion model. The core idea is to represent spectral signatures on a low-dimensional spectral manifold, enabling scene-adaptive modeling of varying endmember (i.e., pure spectral signature) distributions through a two-stream variational network. To further enhance flexibility and robustness, a manifold transformation prior (MTP) is introduced in the latent space to parametrize spectral variations induced by different imaging conditions. The proposed frame jointly estimates varying endmembers and abundance fractions (i.e., subpixel proportions of endmembers) in a unified variational optimization (VO) framework. A comprehensive evaluation on synthetic, semireal, and real-life datasets demonstrates the superior performance and robustness of our method in the HSR task, especially in handling spectral variability.