PHoM: Effective pan-sharpening via higher-order state-space model.

Abstract

Pan-sharpening is intended to generate high-resolution multi-spectral images, utilizing pairs of low-resolution multi-spectral and high-resolution panchromatic images. Recently, the Mamba-based pan-sharpening models achieve state-of-the-art performance due to their efficient long-range relational modeling. However, Mamba inherently obeys a first-order state-space high-dimensional nonlinear mapping, which fails to efficiently encode higher-order expressive interactions of spectral features. In this study, we propose a novel higher-order state-space model for pan-sharpening (PHoM). Our PHoM follows the concept of splitting, interaction, and aggregation for higher-order spatial adaptive interaction and discriminative learning without introducing excessive computational overhead. To model the fusion process between multi-spectral and panchromatic images, we further extend the PHoM into a cross-modal PHoM, which further improves the representation capability by exploiting higher-order cross-modal correlations. We conduct extensive experiments on different datasets. Experimental results show that our method achieves significant performance improvements, outperforming previous state-of-the-art methods on public datasets.