ST-Align: A time series and text alignment framework for cross-subject multivariate time series classification.

Abstract

Multivariate Time Series Classification (MTSC) aims to discern temporal dynamics among variables to classify time series. However, existing MTSC methods predominantly focus on unimodal data, thereby neglecting the rich semantic information embedded within the associated text labels. This limitation hampers the ability to comprehend complex patterns and generalize effectively, ultimately resulting in suboptimal performance in real-world cross-subject scenarios. To address these issues, we propose a time series and text alignment framework (ST-Align) for cross-subject multivariate time series classification. ST-Align leverages the semantic alignment between time series and text labels to improve classification accuracy in semantic spaces, while harnessing the prior knowledge of a large language model (LLM). Specifically, we introduce a two-stage alignment approach. In the first stage, a fine-grained token alignment is employed to capture local information. Inspired by contrastive learning, the captured information is then contrasted with the original text labels to achieve a hybrid Token-Prototype alignment. Extensive experiments on four cross-subject datasets demonstrate that ST-Align achieves state-of-the-art (SOTA) performance on unseen datasets. Moreover, comparative analyses between models with and without the LLM demonstrate their effectiveness, achieving average accuracies of 86.3% and 88.4%, respectively.