Multiscale Spatiotemporal Network With Reinforcement Learning for Predicting Multisensor Systems.

Abstract

To address the modeling challenges posed by multiscale temporal dependencies and sensor spatial correlations in multisensor systems, this article proposes a reinforcement learning-enhanced multiscale spatiotemporal deep network (MSTSDN-RL). Multiscale temporal convolutional networks with causal dilated convolutions are used to capture hierarchical temporal features with enlarged receptive fields, while adaptive graph structure learning and a multibranch graph convolution model capture dynamic sensor correlations. In addition, prediction correction is formulated as a Markov decision process, where reinforcement learning is introduced to sequentially optimize prediction results. Experiments on four C-MAPSS subsets achieve root mean square error (RMSE) values of 12.49, 13.11, 13.58, and 16.37, and Score values of 203.03, 601.25, 205.57, and 1156.88, respectively. On the XJTU-SY dataset, MSTSDN-RL obtains an average RMSE of 29.49 and mean absolute error (MAE) of 25.87. Comparisons with recent state-of-the-art methods, ablation studies, and robustness tests under noise and missing sensor data demonstrate the competitive accuracy, robustness, and generalization capability of the proposed method for aero-engine and rolling bearing prognostics.