Performance of APACHE III models in an Australian ICU.

Abstract

STUDY OBJECTIVE: Evaluation of the performance of the APACHE (acute physiology and chronic health evaluation) III ICU and hospital mortality models at an Australian tertiary adult ICU. DESIGN: Noninterventional, observational study. SETTING: Metropolitan, Australian, tertiary referral medical/surgical ICU. PATIENTS: A total of 3,398 consecutive eligible admissions from January 1, 1995, to December 31, 1997. MEASUREMENTS: Prospective collection of demographic, diagnostic, physiologic, laboratory, admission, and discharge data. RESULTS: The patient sample was younger and more commonly male, with more comorbidities and a different operative and referral source mix, compared to the APACHE III development sample. Receiver operating characteristic curve areas for ICU (0.92) and hospital mortality (0.90) demonstrated excellent discrimination. Observed ICU mortality (9.9%) did not significantly differ from the prediction of the APACHE III model (8.9%) or the APACHE III model adjusted for hospital characteristics (10.5%). The hospital mortality (16.0%) was underestimated by the APACHE III model [13.6%; chi(2)(1) = 7.4; p = 0.01]. With proprietary adjustments for hospital characteristics (14.9%) or referenced to the US database (15.6%), agreement was closer. Good calibration was found with all models except the unadjusted hospital mortality model. CONCLUSION: In contrast to other non-American studies, this Australian study demonstrates that the APACHE III can perform well on independent assessment. As perfect discrimination and calibration cannot coexist in a probabilistic model with dichotomous outcomes, performance of APACHE III models with proprietary adjustment for hospital characteristic provide a good compromise for use in quality surveillance.