The effect of selection bias on the performance of a deep learning-based intraoperative hypotension prediction model using real-world samples from a publicly available database.

Abstract

There are models to predict intraoperative hypotension from arterial pressure waveforms. Selection bias in datasets used for model development and validation could impact model performance. We aimed to evaluate how selection bias affects the predictive performance of a deep learning (DL)-based model and a model using only mean arterial pressure (MAP) as input (MAP-only model). We used the VitalDB open dataset. A hypotensive event was defined as a MAP <65 mm Hg for 1 min. For the 'biased dataset', 'non-hypotensive events' needed to be (a) at the centre of a 'non-hypotensive period' with a MAP of >75 mm Hg for more than 30 continuous minutes and (b) at least 20 min apart from any hypotensive event. For the 'unbiased dataset', all samples were included unless the hypotensive event was already in the input segment. The alarms per hour and positive predictive values were compared between the DL and MAP-only models. The DL model generally performed better than the MAP-only model. For the prediction of intraoperative hypotension 5 min before the event with the DL model, using the unbiased vs the biased testing dataset resulted in 18.1 vs 10.8 alarms per hour (P<0.001) and a positive predictive value of 0.068 vs 0.937 (P<0.001). Both the DL model and the MAP-only model demonstrated worse predictive performance when tested on the unbiased dataset compared with the biased dataset. Although the DL model statistically performed better than the MAP-only model, the difference between the two models was not clinically meaningful. Clinicians should consider the potential impact of selection bias on the validation and the clinical performance of hypotension prediction models. NCT02914444.