CSFformer: Redefining multi-channel time series analysis with cross-scale fusion Transformer.

Abstract

Current Transformer models with channel independence (CI) have made tremendous achievements in time series data analysis. However, the CI methods suffer from short-term fluctuations with intra-channel noise and long-term trend extraction. The fixed receptive field of CI models struggles with capturing multi-scale temporal features within each channel. This paper proposes CSFformer, a cross-scale fusion Transformer for multivariate time series. We first present a Channel-Independent Masking (CIM) module to rectify the feature representation of anomalies and noise in each channel. Then we design a Multi-Scale Pyramid Fusion (MSPF) module, which extracts fluctuation and tread features across various scales. Furthermore, the Multi-Scale Attention Fusion (MSAF) module is introduced for in-depth analysis of interactions between different scales, which significantly contributes to capturing a broader spectrum of complex temporal patterns. We conduct experiments on 7 real-world public datasets. The results show that CSFformer achieves state-of-the-art performance on all datasets, especially in scenarios with obvious fluctuations and trends such as Traffic and Electricity. The source code is available at: https://github.com/damonwan1/CSIformer.