FedMKD: Hybrid Feature Guided Multilayer Fusion Knowledge Distillation in Heterogeneous Federated Learning.

Abstract

In recent years, federated learning (FL) has received widespread attention for its ability to enable collaborative training across multiple clients while protecting user privacy, especially demonstrating significant value in scenarios such as medical data analysis, where strict privacy protection is required. However, most existing FL frameworks mainly focus on data heterogeneity without fully addressing the challenge of heterogeneous model aggregation among clients. To address this problem, this article proposes a novel FL framework called FedMKD. This framework introduces proxy models as a medium for knowledge sharing between clients, ensuring efficient and secure interactions while effectively utilizing the knowledge in each client's data. In order to improve the efficiency of asymmetric knowledge transfer between proxy models and private models, a hybrid feature-guided multilayer fusion knowledge distillation (MKD) learning method is proposed, which eliminates the dependence on public data. Extensive experiments were conducted using a combination of multiple heterogeneous models under diverse data distributions. The results demonstrate that FedMKD efficiently aggregates model knowledge.