Are providers overconfident in predicting outcome after cardiac arrest?

Abstract

To quantify the accuracy of health care providers' predictions of survival and function at hospital discharge in a prospective cohort of patients resuscitated from cardiac arrest. To test whether self-reported confidence in their predictions was associated with increased accuracy and whether this relationship varied across providers. We presented critical care and neurology providers with clinical vignettes using real data from post-arrest patients. We asked providers to predict survival, function at discharge, and report their confidence in these predictions. We used mixed effects models to explore predictors of confidence, accuracy, and the relationship between the two. We completed 470 assessments of 62 patients with 65 providers. Of patients, 49 (78%) died and 9 (15%) had functionally favourable survival. Providers accurately predicted survival in 308/470 (66%) assessments. In most errors (146/162, 90%), providers incorrectly predicted survival. Providers accurately predicted function in 349/470 (74%) assessments. In most errors (114/121, 94%), providers incorrectly predicted favourable functional recovery. Providers were confident (median confidence predicting survival 80 [IQR 60-90]; median confidence predicting function 80 [IQR 60-95]). Confidence explained 9% and 18% of variation in accuracy predicting survival and function, respectively. We observed significant between-provider variability in accuracy (median odds ratio (MOR) for predicting survival 2.93, 95%CI 1.94-5.52; MOR for predicting function 5.42, 95%CI 3.01-13.2). Providers varied in accuracy predicting post-arrest outcomes and most errors were optimistic. Self-reported confidence explained little variation in accuracy.