Multi-Modal Encrypted Retrieval Method with Semantic Feature Fusion towards Internet of Medical Things.

Abstract

There exists a tremendous amount of multimodal data in the Internet of Medical Things (IoMT), retrieval technology can extract target data on demand from the extensive multimodal medical data space, which is crucial for aiding diagnosis and medical informatization. However, existing methods only focus on single-modal data such as medical texts, without considering the privacy protection and retrieval needs of users' multimodal data. Furthermore, these existing methods only match keywords and fail to effectively mine the semantic features of multimodal data, thereby limiting the performance of retrieval systems. To address these issues, this paper proposes a multimodal encrypted retrieval method for the IoMT based on semantic feature fusion and designs a multimodal semantic feature extraction model based on searchable encryption technology to enable encrypted retrieval of multimodal data. Specifically, an edge-cloud collaboration concept is introduced to underpin a secure semantic search architecture tailored for multimodal data, which ensures low-latency encrypted retrieval while safeguarding user privacy. Besides, a semantic-aware multimodal feature extraction method is designed, enhancing the capability of mining semantic features and replacing the traditional keyword retrieval mode with semantic feature retrieval. Moreover, a multimodal data encrypted retrieval method is proposed, employing a block idea and parallel search tree structure, which achieves rapid retrieval of semantic similarity with low-cost and privacy-preserving. Simulation results demonstrate that the proposed method significantly outperforms the latest research regarding precision, search delay, and storage overhead.