Pushing the limits of fluorescence imaging with a restoration neural network aggregating large-view statistics.

Abstract

Deep learning has demonstrated remarkable success in augmenting fluorescence imaging under photon-limited conditions. However, existing restoration networks are typically devised for training with augmented patches far smaller than the full-view raw data, an overlooked aspect that compromises fidelity and noise-resistance due to the loss of global statistics. To address this limitation, we propose a large-patch network (LargePNet), which synergizes the large effective receptive field provided by shallow ultra-large-kernel convolutions and the nonlinear representation capabilities of deep networks through scale separation. It effectively and efficiently leverages large-view global information for restoration. Directly trained with large-view images, LargePNet shows contrasting advantages over state-of-the-art small-patch networks, with 0.5-2 dB higher peak signal-to-noise ratio across eight representative restoration tasks, involving implementations for single-image, video, and volumetric fluorescence data. For full-view processing, LargePNet generally holds around 4-fold and 20-fold higher computational efficiency compared to advanced convolution-based and Transformer-based networks, respectively. The assistance of LargePNet helps achieve 30-hour-long fluorescence imaging to monitor cytoskeleton dynamics, and hour-long tri-color super-resolution imaging to investigate organelle interaction, showcasing its advancement in live-cell imaging.