M³T-CKD: A multi-modal multi-teacher contrastive knowledge distillation framework for survival prediction of patients with carotid atherosclerosis.

Abstract

Patients with carotid atherosclerosis are at risk for cardiovascular disease, which may lead to death. The use of ultrasound image and clinical tabular data to predict the survival of patients is of great value for the treatment and prevention of cardiovascular disease. However, there are two main challenges in survival prediction of patient carotid atherosclerosis: (1) how to effectively fuse ultrasound image and clinical tabular data, and (2) how to accurately segment plaque in ultrasound image and use it for patients survival prediction. To overcome these challenges, we propose a multi-modal multi-teacher contrastive knowledge distillation framework, called M³T-CKD, for survival prediction of patients with carotid atherosclerosis. M³T-CKD leverages teacher network trained on plaque segmentation task to assist student network learning for survival prediction of patient with carotid atherosclerosis. Specifically, we design a modality feature disentanglement (MFD) module for the teacher and student networks to learn shared and specific features of ultrasound image and clinical tabular data to realize fusion. Moreover, we propose a spatial-channel decoupling learning scheme in teacher network to address the issue of low contrast between plaques and surrounding tissues. To further utilize the plaque and background knowledge between the plaque segmentation task and survival task, we propose a novel contrastive knowledge distillation module (CKD). This module encourage the student network to learn plaque features while suppressing the learning of background features for patients survival prediction. We evaluate the performance of M³T-CKD on our collected multi-modal carotid artery ultrasound dataset. Experimental results demonstrate the efficiency of our proposed components and our network achieves the best performance with accuracy of 94.79%, and a p-value < 0.05, outperforming the state-of-the-art methods.