Comprehensive echocardiogram evaluation with view primed vision language AI.

Abstract

Echocardiography is the most widely used cardiac imaging modality, capturing ultrasound video data to assess cardiac structure and function¹. Artificial intelligence (AI) in echocardiography has the potential to streamline manual tasks and improve reproducibility and precision². However, most echocardiography AI models are single-view, single-task systems that do not synthesize complementary information from multiple views captured during a full examination^3,4, and thus lead to limited performance and scope of applications. To address this problem, we introduce EchoPrime, a multi-view, view-informed, video-based vision-language foundation model trained on over 12 million video-report pairs. EchoPrime uses contrastive learning to train a unified embedding model for all standard views in a comprehensive echocardiogram study with representation of both rare and common diseases and diagnoses. EchoPrime then utilizes view classification and a view-informed anatomical attention module to weight video-specific embeddings that accurately map the relationship between echocardiographic views and anatomical structures. With retrieval-augmented interpretation, EchoPrime integrates information from all echocardiogram videos in a comprehensive study and performs holistic clinical interpretation. In datasets from five international independent health-care systems, EchoPrime achieves state-of-the-art performance on 23 diverse benchmarks of cardiac form and function, surpassing the performance of both task-specific approaches and previous foundation models. Following rigorous clinical evaluation, EchoPrime can assist physicians in the automated preliminary assessment of comprehensive echocardiography.