FIND: A Framework for Iterative to Non-Iterative Distillation for Lightweight Deformable Registration.

Abstract

Deformable image registration is crucial for medical image analysis, yet the complexity of deep learning networks often limits their deployment on resource-limited devices. Current distillation methods in registration tasks fail to effectively transfer complex deformation handling capabilities to non-iterative lightweight networks, leading to insignificant performance improvement. To address this, we propose the Framework for Iterative to Non-iterative Distillation (FIND), which efficiently transfers these capabilities to a Non-Iterative Lightweight (NIL) network. FIND employs a dual-step process: first, using recurrent distillation to derive a high-performance non-iterative teacher assistant from an iterative network; second, using advanced feature distillation from the assistant to the lightweight network. This enables NIL to perform rapid, effective registration on resource-limited devices. Experiments across four datasets show that NIL can achieve up to 60 times faster performance on CPU and 89 times on GPU than compared deep learning methods, with superior registration accuracy improvements of up to 3.5 points in Dice scores.