TriAlignNet: A triple-path cross-modality alignment framework for multimodal time series forecasting.

Abstract

Time series forecasting (TSF) is widely applied in critical domains such as power load scheduling, financial risk control, and intelligent manufacturing. However, most existing methods primarily focus on a single numerical modality, making it difficult to fully exploit the multimodal information that is common in real-world scenarios. Recent studies have attempted to incorporate auxiliary modalities, such as text, to enhance forecasting performance. However, existing cross-modal interaction and fusion face significant bottlenecks. Specifically, to address the deep distribution heterogeneity and semantic inconsistency prevalent in multimodal time-series data, this paper breaks through the limitations of traditional single-stage fusion. By adopting a novel perspective of cross-modal progressive alignment, we re-examine multimodal representation learning and formalize the process as a coarse-to-fine continuous modeling problem. In this work, we propose a novel Triple-Path Cross-Modality Alignment Framework (TriAlignNet), which is designed to align multimodal information in TSF. TriAlignNet consists of three complementary alignment pathways: The distribution-level alignment leverages Maximum Mean Discrepancy (MMD) to reduce the statistical gap between text and time-series modalities. The semantic-level alignment introduces a shared anchor matrix as a unified semantic reference, combined with kernel similarity mapping and bidirectional projection mechanisms, to highlight critical semantic dimensions while preserving modality-specific features, thereby strengthening cross-modal consistency and interaction robustness. Finally, the interaction-level alignment employs a multimodal Transformer, where the time series serves as queries and the fused modalities as keys and values, enabling dynamic modeling of fine-grained dependencies and improving forecasting accuracy. Extensive experiments confirm that TriAlignNet enhances time series forecasting by effectively integrating textual information and surpasses existing baselines, offering a robust framework for advancing multimodal alignment in temporal modeling. Our source codes are publicly available at https://github.com/createfree6/TriAlignNet.