Correctformer: A transformer architecture for correcting periodic drift in time-series forecasting.

Abstract

Time-series forecasting is crucial both in production and daily life. Although the Transformer architecture has demonstrated strong potential in this field and significantly improved model performance through continuous iterations of attention mechanism innovations, these advancements have primarily focused on capturing long-range dependencies within sequences. However, insufficient attention has been paid to the inherent periodic patterns in time-series data. Moreover, we observe that applying attention mechanisms can lead to periodic blurring and even periodic drifts, making it challenging for the model to capture the true dynamic patterns of time series, ultimately resulting in degraded forecasting performance. To address this issue, this study proposes Correctformer, a Transformer architecture integrating periodic embedding and periodic correction to enhance the capture of periodic features both before and after the attention mechanism. Specifically, periodic embedding encodes the periodic structural information within time-series data, enabling the model to better perceive and learn periodic characteristics. Periodic correction dynamically adjusts the periodic attributes of the data to rectify periodic drift, restoring the stability of time-series periodicity. Experimental results demonstrate that this approach provides significant advantages in handling data with complex periodic characteristics and offers a more suitable Transformer-based architecture for time-series modeling.