Foundation model embeddings for multimodal oncology data integration.

Abstract

Cancer care generates vast quantities of data including clinical records, pathology images, radiology scans, and molecular profiles, yet these modalities are rarely integrated in a systematic, automated manner within routine clinical workflows, remaining largely siloed across separate departmental and technical systems. Foundation model-driven embeddings—or numerical representations (vectors) that summarize complex data such as text, images, and molecular profiles —offer a framework to integrate these data streams into unified patient representations. Here we examine the HONeYBEE platform’s approach to multimodal integration in oncology, situate it within broader developments in representation learning, and clinical and technical challenges that may shape its path to implementation.