Federated learning with randomized alternating direction method of multipliers and application in training neural networks.

Abstract

Federated learning (FL) is a research area focusing on model training across numerous users while preserving data privacy under the coordination of a central server. The inherent optimization challenges in FL often manifest as nonconvex and nonsmooth problems, presenting significant computational difficulties. This paper proposes a novel FL algorithm that combines the alternating direction method of multipliers (ADMM) with a randomized block-coordinate strategy and general majorization-minimization principle. We provide almost surely subsequential convergence of the generated sequence to a stationary point. We show that our algorithm possesses the best-known complexity bound in terms of the number of communication rounds. Further, through empirical evaluations on well-known datasets, we demonstrate the effectiveness of our algorithm on classification problems using neural networks, underscoring its practical efficacy in real-world applications.