Electronic health record-based prediction models for dementia detection: a systematic review of model performance and quality.

Abstract

Leveraging routine electronic health records (EHR) for dementia detection is a growing field, but quality and clinical utility of existing models are unclear. This systematic review aimed to evaluate performance, methodological quality, and risk of bias of EHR-based dementia prediction models. We systematically searched Medline, EMBASE, Scopus, IEEE Xplore, and ACM from conception until July 2024. All studies and grey literature describing development or validation of probabilistic prediction models using EHR data for dementia detection were included. Risk of bias was assessed using PROBAST. Fifty-six studies (434 prediction models, 155 external validations) were included. Most models were prognostic (66%), used US data (71%), relied solely on structured data, and 47 (11%) were externally validated. Modeled outcomes were extremely heterogeneous: gold-standard clinical criteria were used in 17 models (4%), with others reliant on diagnostic codes for case ascertainment. Discriminative metrics were frequently reported (82% of models), but calibration was rarely assessed (16%). All models were judged high risk of bias, driven by poor outcome definition, inadequate handling of missing data, and potential overfitting. Our review highlights significant issues with methodological rigor and reporting transparency in existing EHR dementia prediction models. Ambiguous outcomes, flawed case ascertainment, and incomplete performance reporting, all limit clinical usefulness. Overall, model performance was difficult to assess and compare across studies due to incomplete reporting. Electronic health record-based dementia prediction is still in its infancy. Methodological rigor and interdisciplinary collaboration are essential to meet clinical needs and achieve real-world impact.