From predictive signal to deployable value: EHR measurement-process features in ICU AKI prediction.

Abstract

Electronic health record (EHR)-based clinical prediction systems increasingly use features that reflect how care is delivered and recorded. We evaluated whether creatinine measurement-process features add incremental, operational, and transportable value for dynamic ICU acute kidney injury (AKI) prediction beyond clinical creatinine trajectories. We performed a retrospective cross-database study using MIMIC-IV for development and temporal testing and eICU-CRD for external validation. Repeated 6-hour landmarks began 24 hours after ICU admission. Predictors were derived from the preceding 24 hours, and the outcome was observed creatinine-defined KDIGO stage 2-3 AKI during the subsequent 24 hours. Clinical creatinine trajectory features were separated from measurement-process features, including count, recency, spacing, density, gaps, and calendar timing. Logistic regression models compared clinical-only, clinical-plus-process, and process-only feature sets. MIMIC-trained preprocessing and coefficients were frozen for eICU validation. The primary metric was patient-equal-weighted Brier score; secondary analyses assessed discrimination, calibration, fixed-burden alerts, redundancy, hospital heterogeneity, and model-class sensitivity. MIMIC-IV included 4571 ICU stays and 41,600 prediction windows; the eICU dynamic validation subset included 23,058 stays and 206,532 windows. Process-only AUROC was 0.560 in MIMIC-IV and 0.681 in eICU, indicating site-dependent marginal signal. Adding process features changed patient-weighted Brier score by + 0.00010 in MIMIC-IV and -0.000136 in eICU. At a fixed 5% alert burden, detection gains were modest. Process features explained 0.16% of residual Brier-loss variance, and conditional permutation changed AUROC by 0.0018. Creatinine measurement-process features captured workflow-related signals but added limited deployable value once clinical creatinine trajectories were represented. EHR process-derived features should be evaluated for incremental performance, calibration, alert burden, redundancy, and transportability before implementation.