Trustworthy Multi-Modal Ultrasound Fusion via Uncertainty Calibration and Conflict Resolution.

Abstract

Multi-modal ultrasound combines tissue information from multiple imaging perspectives, enabling more comprehensive lesion assessment. However, conventional multi-view learning methods typically assume uniform modality quality, ignoring variability caused by imaging noise and patient-specific factors. This oversight limits diagnostic reliability, especially when some modalities provide uncertain or conflicting information. To address this, we identify two key challenges in multi-modal ultrasound fusion: 1) how to quantify modality-wise uncertainty, and 2) how to resolve conflicts among predictions. We propose a novel method, termed TMUF (Trustworthy Multi-modal Ultrasound Fusion), which dynamically integrates information from different modalities through uncertainty calibration and conflict resolution. Specifically, we introduce a cross-modal uncertainty calibration regularizer to estimate evidence-based uncertainty across modalities, aligning uncertainty with prediction correctness. We further develop a credibility-aware fusion strategy that evaluates cross-modal consistency and uncertainty to distinguish credible from non-credible modalities, assigning fusion weights accordingly. We validate TMUF on public and private datasets for breast lesion and liver cancer diagnosis. The proposed method achieves diagnostic accuracies of 88.00% and 92.08%, respectively, outperforming state-of-the-art baselines. These results demonstrate the effectiveness of TMUF in enhancing diagnostic accuracy and robustness for multi-modal ultrasound.