Dual-stream interactive networks with pearson-mask awareness for multivariate time series forecasting.

Abstract

Multivariate time series forecasting (MTSF) aims to predict time series data containing multiple variates, which requires the consideration of both intra-series temporal trends and inter-series interactions. Benefiting from the success of Transformers in computer vision and natural language processing, recently, many time series models are based on the Transformer architecture to reveal intra-series or/and inter-series relationships. However, limited by a single time series embedding representation, most existing methods focus either on the study of changes in the time dimension or on the interactions between multiple variates. There is still a significant research gap in the comprehensive understanding of inter-series interactions and the changes within intra-series in multivariate time series. To bridge this gap, we propose a dual-stream interactive networks with pearson-mask awareness (DSIN-PMA) for MTSF. Specifically, we first employ a dual-stream embedding structure with multivariate embedding and time-step embedding to better represent the diversity in the time and the variate dimensions, respectively. Furthermore, the overall framework of the model adopts a two-stream networks: (1) A cross-multivariate attention with pearson-mask module is proposed to effectively mitigate the impact of noise in multivariate time series data and reduce unnecessary dispersion in cross-variate interactive attention, thereby helping to learn the dependencies between multiple variates more efficiently. (2) A time-step attention module is introduced to learn seasonality and potential trends in the time dimension. Finally, to further enhance the feature representation ability and robustness of the model, we employ a cross-dimension consistency learning strategy to interact with the outputs of the dual-stream encoder. Experimental results on 11 real-world data sets from multiple fields show that the DSIN-PMA model significantly outperforms the baseline model, and achieves 5.12%-17.43% improvements over state-of-the-art (SOTA) methods. In-depth analysis shows that the strategy that comprehensively considers both the variate dimension and the time-step dimension outperforms the single-dimensional strategy. Our source codes are publicly available at https://github.com/yejunjiePhD/DSIN-PMA.