BPGAN: Bidirectional CT-to-MRI prediction using multi-generative multi-adversarial nets with spectral normalization and localization.

Abstract

Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) are widely used detection technology in screening, diagnosis, and image-guided therapy for both clinical and research. However, CT imposes ionizing radiation to patients during acquisition. Compared to CT, MRI is much safer and does not involve any radiations, but it is more expensive and has prolonged acquisition time. Therefore, it is necessary to estimate one modal image from another given modal image of the same subject for the case of radiotherapy planning. Considering that there is currently no bidirectional prediction model between MRI and CT images, we propose a bidirectional prediction by using multi-generative multi-adversarial nets (BPGAN) for the prediction of any modal from another modal image in paired and unpaired fashion. In BPGAN, two nonlinear maps are learned by projecting same pathological features from one domain to another with cycle consistency strategy. Technologically, pathological prior information is introduced to constrain the feature generation to attack the potential risk of pathological variance, and edge retention metric is adopted to preserve geometrically distortion and anatomical structure. Algorithmically, spectral normalization is designed to control the performance of discriminator and to make predictor learn better and faster, and the localization is proposed to impose regularizer on predictor to reduce generalization error. Experimental results show that BPGAN generates better predictions than recently state-of-the-art methods. Specifically, BPGAN achieves average increment of MAE 33.2% and 37.4%, and SSIM 24.5% and 44.6% on two baseline datasets than comparisons.