HomoDiff: Noise-Robust Multi-modal Homography Estimation via Diffusion Model.

Abstract

Multi-modal homography estimation aims to align multi-modal images captured by different sensors. Existing methods are largely confined to clean images, neglecting the noise degradation in real-world imaging scenarios. To tackle this issue, we propose a novel diffusion-based homography estimation framework namely HomoDiff, for noise-robust multi modal homography estimation. The design of our network is motivated by an interesting finding, revealing that stronger denoising does not always result in higher registration accuracy. Inspired by this finding, we formulate the registration process in a novel diffusion-based architecture. Specifically, in each inference step, a Multi-modal Image Denoising (MID) subnet is designed to remove the noise with the multi-modal guidance, followed by a Multi-modal Homography Estimation (MHE) subnet to predict the homography matrix at the current denoising intensity. Finally, an Adaptive Homography Selection (AHS) subnet is used to select the optimal homography matrix from all inference steps with different denoising intensities. Through this selection strategy, we can avoid over-denoising to maintain consistently high registration accuracy across different noise levels. To the best of our knowledge, our HomoDiff is the first attempt to address the multi-modal homography estimation problem with noise degradation, which can handle varying noise levels with a single unified framework. Extensive experimental results show that our HomoDiff achieves high registration accuracy on various multi-modal datasets with different noise levels. The source codes are available at https://github.com/Langweng/HomoDiff.git.