Evaluating short-term forecast accuracy across COVID-19 waves using penalized spline models.

Abstract

During the COVID-19 pandemic, one of the key objectives was to provide daily information on the evolution of the disease. The aim of this study is to evaluate whether the modelling approach used in our area to provide short-term forecasts was accurate enough to detect different phases of the epidemic. To support short-term predictive analyses, our health provider deployed advanced IT infrastructure, including real-time monitoring systems and the Oracle Analytics Server, to aggregate and analyse epidemiological data from multiple sources. The study period ran from February 15, 2020 to September 15, 2022. We employed penalized regression splines to evaluate daily counts of SARS-CoV-2 positive cases, hospitalizations, and Intensive Care Unit admissions. To address overdispersion in the data, we used the Negative Binomial distribution. A generalized additive model was applied to account for patterns in the data. Three types of penalties (m₁-m₃) were used for the first and second derivatives of this model for 2- and 5-day predictions. Prediction errors were evaluated using the root median square error and relative error. A daily median of 437.5 cases were recorded. Hospital admissions peaked at 314, with ICU admissions peaking at 39. Prediction models showed that m₁ and m₂ best fit the observed data, especially during quarantine and the second state of emergency, where success rates exceeded 70-80 %. However, during the Omicron wave, accuracy decreased, with success rates for positive cases ranging from 50 % to 74 % in 5-day predictions. The models performed reasonably well in predicting daily ICU admissions, but not for hospital admissions and were worse in forecasting positive cases. The highest errors were observed during sudden changes in trend. These findings underscore the importance of health care information systems that provide reliable and updated information on daily bases to develop predictions to help clinicians and managers.