External validation and application of risk prediction model for ventilator-associated pneumonia in ICU patients with mechanical ventilation: A prospective cohort study.

Abstract

Early identification and prevention of ventilator-associated pneumonia (VAP) in patients with mechanical ventilation (MV) through reliable prediction model undergoing a rigorous and standardized process is essential for clinical decision-making. This study aims to externally validate the VAP prediction model previously developed by a tertiary hospital in Northwestern China, using data from different time periods or hospitals, and to develop a web-based model calculator for clinical application to evaluate the model's prediction performance and generalizability. We prospectively collected MV patients data from the ICUs of two tertiary hospitals in Northwestern China for external validation of the model. Temporal and geographical validation were performed at the hospital where the model was developed and another hospital, respectively. The area under the receiver operating characteristic curve (AUC), Howsmer-Lemeshow test, calibration curve and decision curve analysis (DCA) were used to evaluate the model's discrimination, calibration and clinical applicability, respectively. A web-based model calculator was further developed and applied to MV patients in one of the hospitals to obtain the prediction probabilities of VAP risk. Model performance was evaluated using a confusion matrix and diagnostic tests. The temporal and geographical validation cohorts included 416 and 410 patients, and the AUCs were 0.814 and 0.800, respectively. The Hosmer-Lemeshow tests (both P > 0.05) and calibration curves showed a relatively high consistency. The DCA revealed the model threshold probabilities in the temporal (2.0 % to 50.0 %) and geographical validation (5.0 % to 70.0 %). The web-based model calculator (https://vapnomogram.shinyapps.io/VAPDynNomapp/) was applied to 202 patients in clinical practice. The cut-off value of the prediction probability was 0.096, with an accuracy of 0.911, a sensitivity of 0.900, a specificity of 0.912, and a positive and negative predictive value of 0.529 and 0.988, respectively. The VAP prediction model showed relatively stable and relaible clinical prediction performance and generalizability, with a clinical application and promotion value.