Decoupling Neural Networks to Leverage Uniform Representation and Balance Personalization and Collaboration in Federated Learning.

Abstract

Federated learning (FL), a distributed learning paradigm focused on preserving data privacy, faces challenges due to varying data distributions among clients, impacting global model performance. To mitigate data heterogeneity, we propose FedUB-a personalized FL framework leveraging uniform feature representation and balancing personalization and collaboration in the classifier. Specifically, the uniform representation (UR) in FedUB provides all clients with a shared feature extractor and a common representation centroid (RC). Achieving this uniformity involves incorporating a regularization term to reduce the gap between global and local RCs. Additionally, an importance estimation of the parameters in the classifier is provided to partition the parameters into two parts: the personalized component and the collaborated component. Specifically, the personalized component adapts to local data, while the collaborated component prevents the classifier from overfitting local data. Theoretically, we establish the existence of the UR, demonstrating its effectiveness in reducing the average generalization bound. Experiments on benchmark datasets consistently demonstrate the performance gains and improved generalization behavior of FedUB.