HepaCopilot: A 6G-Enabled Multimodal Vision-Language Agent for Real-Time Hepatocellular Carcinoma Risk Stratification via Contrast-Enhanced Ultrasonography with Chain-of-Thought Clinical Reasoning.

Abstract

Hepatocellular carcinoma (HCC) is one of the leading causes of cancer-related deaths around the world. One major barrier to detecting HCC is that CEUS is heavily reliant on operator interpretation, leading to an excessive amount of variability in how the test results are interpreted across different clinical settings. We developed HepaCopilot, which is a multimodal, vision-language agent that is able to integrate a sequence of temporal CEUS videos together with clinical metadata, thus providing an interpretable risk assessment of HCC using a structured method of clinical reasoning using Chain-of-Thought (CoT). HepaCopilot's hierarchical temporal encoding architecture, along with cross-modal attention mechanisms, allows the systematic extraction of features in each of the three phases: the arterial, portal venous, and delayed phases. We performed an evaluation of HepaCopilot using a publicly available dataset from TCIA B-mode-and-CEUS-Liver comprising 120 subjects who had CEUS examinations and who had been pathologically diagnosed with HCC. Using a subject-level evaluation of a test set containing 18 subjects we show that our method achieves a discrimination performance of AUROC = 0.94 (bootstrap 95% CI: 0.82-1.00, 2,000 resamples) when compared to baseline methods, though the wide confidence interval reflects the limited test set size.