CoCoFR: Collaborative codebooks learning with soft matching strategy for blind face restoration.

Abstract

Blind Face Restoration (BFR) has garnered considerable attention for its practical applicability to recover high-quality (HQ) facial images from their degraded versions. Existing BFR methods primarily incorporate diverse priors to mitigate its ill-posed nature. Notably, the codebook prior, which aggregates facial representations from HQ images has achieved impressive results. However, two performance constraints remain: i) The reliance on a single spatial-domain codebook neglects the potential information in the frequency domain. ii) The commonly used feature-matching strategies overlook the valid information encapsulated within the low-quality (LQ) identity features. To address these issues, we propose CoCoFR, which learns collaborative codebooks in both spatial and frequency domains and implements adaptive matching between LQ and HQ features with a designed Dual Codebooks Cross Attention (DCCA) module. Additionally, benefiting from its global receptive fields and linear complexity, CoCoFR facilitates coarse-to-fine feature fusion via a simple yet effective state space model (Mamba)-based fusion block MFB. Extensive experiments on both synthetic and real-world datasets validate the superiority of our CoCoFR in terms of realness and fidelity compared to state-of-the-art methods.