Do machine learning methods make better predictions than conventional ones in pharmacoepidemiology? A systematic review, meta-analysis, and network meta-analysis.

Abstract

To synthesize existing evidence and compare the predictive performance of conventional statistical (CS) models versus machine learning (ML) methods in pharmacoepidemiology. Medline, Embase, PsycINFO, CINAHL and Web of Science databases were systematically searched for predictive pharmacoepidemiologic studies published between January 2018 and September 2025. Independent reviewers extracted predictive metrics and other data from each study and assessed the quality of the comparison between methods. The relative performance of ML compared to CS was estimated for each prediction objective. Performance metrics were pooled in meta-analyses and Bayesian network meta-analyses (NMA). Among 9106 records identified, 65 studies met inclusion criteria, encompassing 83 prediction objectives. For 84 % of these objectives, CS was outperformed by at least one ML method. The median sample size across these studies was 2691 subjects (50 to 1,807,159), and, for binary outcomes, the median number of events per candidate predictor was 17.9 (range 0.28 to 24,260). We observed a risk of bias in the comparison according to at least one of eight major criteria for 39 prediction objectives (47 %). The pooled area under the receiver-operator curve (AUC) ratio for the highest-performing ML method in studies with low risk of bias was estimated as 1.07 (95 % confidence interval 1.03-1.12) in favor of ML, but with very high heterogeneity. NMA of 197 comparisons estimated an AUC ratio of 1.07 (95 % credible interval 1.04-1.12) for boosted methods compared with logistic regression and ranked Gradient Boosting Machine and XGBoost consistently among the best-performing methods. Machine learning methods applied to structured pharmacoepidemiologic data demonstrated a consistent yet modest advantage in discriminative performance relative to conventional statistical models. This advantage was most evident for boosted methods such as GBM and XGBoost. However, greater rigor in reporting methodological details is recommended to improve the comprehension, transparency, and reproducibility of studies. PROSPERO 2023 registration number: CRD42023426986.