Topology-Optimal Multiple Gossip Steps for Decentralized Federated Learning via Gossip Tensor.

Abstract

Decentralized federated learning (DFL) has gained significant attention as a framework for analyzing large-scale data distributed across multiple sites, where communication between sites is constrained by a decentralized graph structure. Due to privacy concerns and high communication costs, reducing the number of communication rounds in DFL has become an important area of research. This article investigates the effects of the decentralized graph's topology on the convergence rate of DFL algorithms and introduces a novel tensor-based multiple-gossip-steps (T-MGS) method to optimize communication efficiency from the topology aspect. The core idea of this method is to use gossip tensors to guide the information flow between sites and enable dynamic adjustments to the transmitted content at each communication step without increasing its volume. The proposed method minimizes the second-largest absolute eigenvalue of the equivalent gossip matrix, a key factor influencing convergence speed. Experimental results on both simulated and real datasets demonstrate that the proposed T-MGS outperforms existing strategies in terms of communication efficiency, reducing the number of communication rounds without compromising model accuracy.