Uncertainty-Guided Discriminative Priors Mining for Flexible Unsupervised Spectral Reconstruction.

Abstract

Existing supervised spectral reconstruction (SR) methods adopt paired RGB images and hyperspectral images (HSIs) to drive the overall paradigms. Nonetheless, in practice, "paired" requires higher device requirements such as specific well-calibrated dual cameras or more complex and exact registration processes among images with different time phases, widths, and spatial resolution. To tackle the above challenges, we propose a flexible uncertainty-aware unsupervised SR paradigm, which dynamically establishes the forceful and potent constraints with RGBs for driving unsupervised learning. As a specific plug-and-play tail in our paradigm, the uncertainty-aware saliency alignment module (USAM) calculates pixel- and spectralwise information entropy for uncertainty estimation, which attempts to represent the corresponding reflectivity or radiance to the light among different objects in various scenes, forcing the paradigm to adaptively explore the scene-agnostic prominent features. Furthermore, a progressively parallel network under our unsupervised paradigm is conducted to excavate discriminate structural and semantic priors of RGBs to assist in recovering dependable HSIs: 1) a learnable rank-guided structural representation (LRSR) flow is leveraged to characterize the latent structural priors via excavating nonzero elements in the full-rank matrix and further preserve evident boundaries in HSIs; and 2) a coarse-to-fine bandwise semantic perception (CBSP) flow is conducted to propagate perceptual bandwise affinity for aggregating and strengthening intrinsic interband dependencies, and further extract delicate semantic priors, which can recover plentiful contiguous spectral information in HSIs. Comprehensive quantitative and qualitative experimental results on three visual and two remote sensing benchmarks have shown the superiority and robustness of our method. We also conducted nine existing SR methods in our unsupervised paradigm to recover HSIs without any manual intervention, which proves the generality of our paradigm to some extent.