Heterogeneity-aware high-efficiency federated learning with hybrid synchronous-asynchronous splitting strategy.

Abstract

Federated Learning (FL) offers a promising privacy-preserving framework for collaborative global model training without exposing local private data. However, system heterogeneity in FL causes the straggler issue, where resource-limited edge devices delay global model aggregation. Existing approaches, such as asynchronous mechanisms or kick-out methods, primarily focus on optimizing model convergence efficiency but often overlook edge resource constraints, potentially resulting in model bias toward high-performance devices or omission of critical data. To cope with this, we propose HA-HEFL, a novel Heterogeneity-Aware High-Efficiency Federated Learning framework to balance training efficiency, model accuracy, and resource consumption. HA-HEFL features a resource-aware adaptive model customization mechanism that dynamically tailors suitable model architectures based on device capabilities using neuron-level profiling and priority-based selection methods. Additionally, HA-HEFL employs a hybrid synchronous-asynchronous split training strategy, dividing into synchronous edge feature extraction and asynchronous global classifier updates based on knowledge distillation, thereby enhancing training efficiency and model performance. A baseline-prioritized weighted aggregation scheme is adopted to further ensure balanced global model updates. Extensive experiments on three real-world datasets demonstrate that HA-HEFL significantly improves convergence speed, model accuracy, and reduces network traffic compared to state-of-the-art methods.