Evaluation of the Clinical Utility of the Bone Metastases Ensemble Trees for Survival Decision Support Platform (BMETS-DSP): A Case-Based Pilot Assessment.

Abstract

The Bone Metastases Ensemble Trees for Survival Decision Support Platform (BMETS-DSP) provides patient-specific survival predictions and evidence-based recommendations to guide multidisciplinary management for symptomatic bone metastases. We assessed the clinical utility of the BMETS-DSP through a pilot prepost design in a simulated clinical environment. Ten Radiation Oncology physicians reviewed 55 patient cases at two time points: without and then with the use of BMETS-DSP. Assessment included 12-month survival estimate, confidence in and likelihood of sharing estimates with patients, and recommendations for open surgery, systemic therapy, hospice referral, and radiotherapy (RT) regimen. Paired statistics compared pre- versus post-DSP outcomes. Reported statistical significance is <i>P</i> < .05. Pre- versus post-DSP, overestimation of true minus estimated survival time was significantly reduced (mean difference -2.1 [standard deviation 4.1] <i>v</i> -1 month [standard deviation 3.5]). Prediction accuracy was significantly improved at cut points of < 3 (72 <i>v</i> 79%), ≤ 6 (64 <i>v</i> 71%), and ≥ 12 months (70 <i>v</i> 81%). Median ratings of confidence in and likelihood of sharing prognosis significantly increased. Significantly greater concordance was seen in matching use of 1-fraction RT with the true survival < 3 months (70 <i>v</i> 76%) and < 10-fraction RT with the true survival < 12 months (55 <i>v</i> 62%) and appropriate use of open surgery (47% <i>v</i> 53%), without significant changes in selection of hospice referral or systemic therapy. This pilot study demonstrates that BMETS-DSP significantly improved physician survival estimation accuracy, prognostic confidence, likelihood of sharing prognosis, and use of prognosis-appropriate RT regimens in the care of symptomatic bone metastases, supporting future multi-institutional validation of the platform.