Systematically decoding pathological morphologies and molecular profiles with unified multimodal embedding.

Abstract

Systematic cross-modality inference and integration of pathological morphologies and multilayer molecular profiles have advanced disease biology; however, methodological challenges remain in multimodal learning. Here, we present Multi-Embed, a unified and interpretable framework for multimodal learning between multilevel morphologies and multilayer molecular profiles. Multi-Embed achieves superior performance in morphology-molecule inference and integration, fine-grained tissue architecture identification and spatiotemporal trajectory modeling across diverse benchmark tasks, underscoring its utility for enhancing our understanding of disease pathogenesis.