Improved Residual Learning in Diffusion Models.

Abstract

Diffusion models (DMs) have profoundly transformed the field of generative modeling, delivering exceptional image generation quality. Nevertheless, residuals in the generated images, arising from modeling errors that accumulate along the practical sampling trajectories of pre-trained DMs, remain an inevitable challenge. To mitigate this, we propose a novel residual learning framework built upon a parameterized correction function, designed to improve modeling performance through residual correction. Most notably, our framework exhibits remarkable transferable residual correction capabilities, enabling a correction function optimized for a specific pre-trained DM on a given dataset to enhance the performance of other DMs trained on the same dataset. To further improve our framework, we introduce an advanced training approach that designs an ODE sampler bank to refine residual simulation and incorporates a score consistency maintenance technique to enhance model convergence. Building on this approach, the optimized correction function achieves substantial improvements in residual correction. Extensive experiments performed on four widely used datasets and multiple pre-trained DMs validate the effectiveness and superiority of our residual learning framework.