Multimodal Knowledge Graph Completion by Cross-Modal Interaction With Similarity Enhancing and Difference Embracing.

Abstract

Multimodal knowledge graph completion (MMKGC) enhances the precision and breadth of application of knowledge graphs by integrating rich data from various modalities, steadily increasing its appeal in the research community. Prior studies mainly focus on the common representation of different modalities while neglecting the different and complementary features. On the contrary, some works tend to model triples of each modality separately while overlooking the similarities between modalities. It is challenging to associate the heterogeneous modalities effectively for MMKGC. In this article, we introduce a novel MMKGC framework by cross-modal interaction with similarity-enhancing and difference-embracing (CISEDE), which leverages both the similarities and differences among multimodal entities by a proposed cross-modal interaction mechanism. In the cross-modal interaction, multihead attention is employed to enhance similarity information from multimodal entities and embrace different information by linking various modal triples. Through relation-guided fusion, the modal triples are decoded and merged for MMKGC. The experimental results on three commonly used datasets, FB15k-237, WN9, and WN18RR, show that the proposed method achieves state-of-the-art performance.