Probingformer: Modeling cross-dimension dependencies via pretraining-probing for long-term time series forecasting.

Abstract

Recent advances in multivariate time series forecasting (MTSF) have been driven by supervised learning and pretraining-fine-tuning paradigms, while pretraining-probing has received limited attention due to weaker empirical results. In this work, we revisit pretraining-probing as a principled approach to decouple representation learning from forecasting, which helps mitigate overfitting to spurious correlations commonly found in channel-dependent methods. Building on this, we propose Probingformer, a framework specifically designed to improve MTSF performance with a focus on explicitly capturing variable correlations. This framework leverages pretraining to learn rich temporal and variable interactions, enabling forecasting through a lightweight probing step. To further strengthen structural modeling, we introduce the Cross-Temporal Dependency Module (TDM) and the Cross-Dimension Dependency Module (DDM), responsible for capturing intra-sequence patterns and inter-variable correlations, respectively. Extensive experiments on ten datasets with diverse correlation structures demonstrate that Probingformer achieves state-of-the-art performance. Further analysis shows that, as a channel-dependent method, it accurately captures inter-variable correlations while effectively mitigating overfitting.