Optical-Cue-Guided Diffusion Probabilistic Model for Reflection Removal.

Abstract

This article presents a novel reflection removal algorithm that integrates a flash-based optical cue into a diffusion model to control the recovery of the transmission image. The algorithm accepts a pair of ambient and flash images as inputs, and a flash-only image, which corresponds to the one captured with flash as the sole illumination source, is derived from the inputs. In light of the reflection-free nature of the flash-only image, we use it to guide the diffusion model to reconstruct the structures of the transmission image. A feature distillation scheme is designed to infer the chromatic attributes of the transmission image from the ambient image, and the features are used to modulate the generative priors learned by the diffusion model. We use time-aware strategies to ensure the synchronization between feature distillation and the dynamic image generation process of the diffusion model. The performance of the proposed algorithm is sequentially optimized in latent and pixel spaces. We also develop a plug-and-play fidelity-enhancing module (FEM) and integrate it into the proposed model to enable the faithful reconstruction of fine-granular visual characteristics of the target scene and reduce artifacts. Comparative experiments demonstrate that the proposed algorithm shows superior quantitative and qualitative performance over state-of-the-art methods in real-world scenarios. By leveraging the optical cue and the generative capability of the diffusion model, the algorithm can accurately restore the visual details of the transmission image even in the presence of strong reflections, and it also exhibits satisfactory robustness against nonlinear image representation and misalignment.