AdaptiveFL: Communication-Adaptive Federated Learning Under Dynamic Bandwidth.

Abstract

Federated learning (FL) is a distributed machine learning paradigm that enables heterogeneous devices to train a model collaboratively. Recognizing communication as a bottleneck in FL, existing communication-efficient solutions, e.g., HeteroFL and LotteryFL, etc., utilize gradient sparsification to reduce communication costs. However, existing solutions fail to address the dynamic bandwidth issue in which the bandwidth of each client is constantly changing throughout the training process. In this article, we propose AdaptiveFL, a communication-adaptive FL framework, considering the dynamic constraints of bandwidth. The design of AdaptiveFL follows two key steps: 1) in each round, each device selects a best-fit sub-model for communication per currently available bandwidth; and 2) to guarantee the performance of each sub-model sent under dynamic bandwidth constraints, AdaptiveFL employs a local training method that enables each device to train a "tailorable" local model, which can be tailored to any sparsity with competitive accuracy. We compare AdaptiveFL with several communication-efficient SOTA methods and demonstrate that AdaptiveFL outperforms other baselines by a large margin.