DIFF-MF: A Difference-Driven Channel-Spatial State Space Model for Multimodal Image Fusion.

Abstract

Multimodal image fusion aims to integrate complementary information from multiple source images to produce high-quality fused images with enriched content. Although existing approaches based on state space models (SSMs) have achieved satisfactory performance with high computational efficiency, they tend to either over-prioritize infrared intensity at the cost of visible details, or conversely, preserve visible structure while diminishing thermal target salience. To overcome these challenges, we propose DIFF-MF, a novel difference-driven channel-spatial SSM for multimodal image fusion. Our approach leverages feature discrepancy maps between modalities to guide feature extraction, followed by a fusion process across both channel and spatial dimensions. In the channel dimension, a channel-exchange module enhances channel-wise interaction through cross-attention dual state space modeling, enabling adaptive feature reweighting. In the spatial dimension, a spatial-exchange module employs cross-modal state space scanning to achieve comprehensive spatial fusion. By efficiently capturing cross-modal discrepancy features and integrating them in a well-balanced manner, DIFF-MF effectively fuses complementary multimodal information. Experimental results on the driving scenarios and low-altitude unmanned aerial vehicle (UAV) datasets demonstrate that our method outperforms existing approaches in both visual quality and quantitative evaluation.