DH-MSVM: A hybrid algorithm for seeking quality support vectors in distributed learning.

Abstract

Data heterogeneity is a common yet complex challenge in distributed machine learning scenarios. However, current Distributed Support Vector Machines (DSVMs) lack effective mechanisms to identify suitable support vectors across diverse data structures, limiting their ability to dynamically adjust decision boundaries. To address this issue, we propose a Distributed Hybrid Learning based on Support Vector Machine (DH-SVM). This approach leverages global pre-learning to capture data structure information, which then guides local learning processes to identify higher-quality support vectors and adaptively refine decision boundaries. Furthermore, considering the computational overhead inherent in distributed learning, we enhance our algorithm by incorporating a Markov sampling technique (DH-MSVM). Theoretically, we derive the generalization bound of the algorithm based on uniformly ergodic Markov chain samples and establish a fast learning rate, demonstrating the robustness and scalability of DH-MSVM. Empirically, extensive experiments on real-world datasets validate the superior performance of the proposed algorithms.