A link prediction method for multi-modal knowledge graphs based on Adaptive Fusion and Modality Information Enhancement.

Abstract

Multi-modal knowledge graphs (MMKGs) enrich the semantic expression capabilities of traditional knowledge graphs by incorporating diverse modal information, showcasing immense potential in various knowledge reasoning tasks. However, existing MMKGs encounter numerous challenges in the link prediction task (i.e., knowledge graph completion reasoning), primarily due to the complexity and diversity of modal information and the imbalance in quality. These challenges make the efficient fusion and enhancement of multi-modal information difficult to achieve. Most existing methods adopt simple concatenation or weighted fusion of modal features, but such approaches fail to fully capture the deep semantic interactions between modalities and perform poorly when confronted with modal noise or missing information. To address these issues, this paper proposes a novel framework model-Adaptive Fusion and Modality Information Enhancement(AFME). This framework consists of two parts: the Modal Information Fusion module (MoIFu) and the Modal Information Enhancement module (MoIEn). By introducing a relationship-driven denoising mechanism and a dynamic weight allocation mechanism, the framework achieves efficient adaptive fusion of multi-modal information. It employs a generative adversarial network (GAN) structure to enable global guidance of structural modalities over feature modalities and adopts a multi-layer self-attention mechanism to optimize both intra- and inter-modal features. Finally, it jointly optimizes the losses of the triple prediction task and the adversarial generation task. Experimental results demonstrate that the AFME framework significantly improves multi-modal feature utilization and knowledge reasoning capabilities on multiple benchmark datasets, validating its efficiency and superiority in complex multi-modal scenarios.