Multicenter Prospective Validation of an Updated Proprietary Sepsis Prediction Model.

Abstract

The Epic Sepsis Model version 2 (ESM v2) is a widely implemented proprietary sepsis prediction model, but no multicenter, external validation of its performance has been reported to guide adoption and use. To conduct a multicenter validation of the ESM v2 to compare performance against the original ESM v1, outline differences across heterogenous clinical sites, and compare model performance against clinician recognition of sepsis. This prognostic study included adult inpatient encounters at 4 large US health systems between August 31, 2023, and March 11, 2025. At each site, data were collected for a consecutive period immediately following new model implementation. Data were analyzed from July 23 to August 19, 2025. Sepsis was defined using Sepsis-3 clinical consensus criteria. Model discrimination was assessed using area under the receiver operating characteristic curve (AUROC) at the encounter level and prediction level with 4-hour, 12-hour, and hospitalization-wide time horizons. Performance against clinician recognition of sepsis was measured using antibiotics, lactate, and body culture orders. Of 227 091 inpatient encounters, 7401 (3.3%; median [IQR] age, 65 [54-75] years; 3359 [45.4%] female; 2.7% Asian; 24.6% Black; 64.6% White; 7.1% Hispanic ethnicity) met sepsis criteria and 219 690 (96.7%; median [IQR] age, 48 [33-65] years; 123 563 [56.2%] female; 2.5% Asian; 38.8% Black; 49.6% White; 9.6% Hispanic ethnicity) did not. At the encounter level, the AUROC ranged from 0.82 (95% CI, 0.81-0.83) to 0.92 (95% CI, 0.92-0.93) across study sites. At the prediction level with a 12-hour time horizon, the AUROC ranged from 0.75 (95% CI, 0.74-0.75) to 0.85 (95% CI, 0.85-0.85). Comparison against clinician recognition of sepsis yielded a minor decrease in performance, with the resulting encounter-level AUROC ranging from 0.80 (95% CI, 0.79-0.81) to 0.90 (95% CI, 0.89-0.90) across sites. Positive predictive values remained low, from 0.13 (95% CI, 0.13-0.14) to 0.26 (95% CI, 0.25-0.27), with a high number needed to evaluate and high alert burden. In this prognostic study of a new sepsis prediction model, a multicenter prospective validation performed across 4 major US health systems found improved discrimination for the early prediction of sepsis but noted high institutional variability, low positive predictive value, and high alert burden.