Granger-TSllm: Granger causality enhanced LLMs with residual-quantized tokenizer for multivariate time series forecasting.

Abstract

Multivariate time series (MTS) forecasting is essential for process monitoring and decision optimization in numerous real-world domains. Existing deep models need to be trained from scratch on abundant data, resulting in limited generalization. Inspired by the success of large language models (LLMs) on various cross-modality tasks, recent research introduces LLMs for MTS forecasting. However, due to the inherent modality gap between MTS and text, there remain two challenges in generating MTS representations suitable for LLMs and handling the intricate inter-variable dependencies. To tackle these challenges, we propose a novel two-stage framework utilizing LLMs for MTS forecasting, called the Granger causality enhanced LLM with residual-quantized tokenizer (Granger-TSllm). Specifically, we first design a deep Residual-Quantized Time Series Tokenizer, which generates compact discrete MTS embeddings natural for LLMs via compression and quantization. Next, we selectively fine-tune an LLM to fully leverage its pretrained representation learning capability for complex temporal modeling and forecasting. In particular, we introduce an interpretable Granger Causality Modification Module to enhance the prediction performance of LLM based on inter-variable nonlinear Granger causality correlations. Our comprehensive evaluations demonstrate that Granger-TSllm is a powerful MTS forecaster that outperforms the mainstream state-of-the-art models. Moreover, Granger-TSllm exhibits robust generalization capabilities, excelling in both few-shot and zero-shot forecasting scenarios.