A Personalized and Privacy-Preserving Federated Transformer Framework for Multilingual Sentiment Analysis.

Abstract

Personalized federated learning for multilingual sentiment analysis poses significant challenges arising from linguistic heterogeneity, non-IID data distributions, and strict privacy requirements. This paper proposes FedPerX, a federated transformer framework that integrates residual adapter-based personalization with adaptive multi-granular differential privacy. The architecture leverages a frozen multilingual backbone (XLM-R) while enabling each client to train lightweight, client-specific adapters. Privacy is enforced through dynamic noise injection at both the feature and adapter levels, calibrated using gradient sensitivity. FedPerX is evaluated on two multilingual benchmarks-MARC and TSMD-spanning structured reviews and informal social media content across more than ten languages. Experimental results demonstrate consistent improvements over seven state-of-the-art baselines, with up to +4.3% gains in macro-F1, a 70% reduction in communication overhead, and the lowest variance in client-level performance. Comprehensive analyses, including fairness, personalization gap, privacy-utility trade-off, and ablation studies, validate the framework's robustness and adaptability. FedPerX advances the design of scalable, personalized, and privacy-preserving models for federated multilingual sentiment analysis.