A Quaternion Rotation-Enhanced Differential Privacy Framework for Image Privacy Protection.

Abstract

With the widespread application of deep learning in various scenarios, the user data processed by deep neural networks during training and inference often contains sensitive information. Directly exposing this data may pose privacy risks. To prevent the leakage of user data privacy, previous research has proposed privacy-preserving schemes that use encrypted images for training and inference. However, existing schemes struggle to balance privacy protection with model accuracy and show limited adaptability across various visual tasks. To address these issues, this article presents a novel privacy-preserving scheme, quaternion rotation-enhanced differential privacy (QRDP), along with a dedicated neural network, the quaternion Fourier Transformer (QFT). Specifically, QRDP utilizes quaternion rotation operations to map the RGB representation of an image into quaternion space, generating an encrypted quaternion representation. Then, QRDP adds noise to the encrypted representation within a differential privacy (DP) framework, thereby providing theoretical privacy guarantees. QFT, in turn, extracts robust global feature representations from encrypted images through a quaternion patch embedding module and a Fourier feed-forward layer, meeting the requirements of various visual tasks. Through subjective and objective image quality assessments and corresponding attack experiments, this article verifies the security of QRDP. In tasks such as encrypted image classification and the more challenging cross-modal retrieval, the proposed scheme demonstrates outstanding performance, with both performance and security exceeding those of existing schemes.