Evaluation of the Applicability of Synthetic Data in the Development of Colorectal Cancer Survival Prediction Models: External Validation of Advanced Machine Learning Models Based on National Cancer Data Center Data.

Abstract

Limited data availability and privacy constraints hinder the development of robust survival prediction models for personalized treatment. Synthetic data offers a promising solution, preserving the statistical properties of real clinical data. This study aimed to quantitatively assess the feasibility of using synthetic data for survival prediction by evaluating model transfer performance to real-world hospital data, with a focus on model transfer strategies. We developed and validated colorectal cancer survival prediction models using the National Cancer Data Center (NCDC) synthetic data (30,683 patients from 3 Korean institutions) for pretraining and real hospital data (2170 patients from Hwasun Jeonnam University Hospital) for external validation. We evaluated 3 model transfer strategies-domain adaptation, zero-shot, and ensemble-using extreme gradient boosting (XGBoost) and light gradient boosting machine (LightGBM). In total, 48 model configurations were tested, defined by the combination of algorithms (LightGBM and XGBoost), sampling technique (no-sampling, random undersampling [RUS], and synthetic minority oversampling technique combined with edited nearest neighbors [SMOTEENN]), model type (baseline, domain adaptation, zero-shot, and ensemble), and optimization objective (area under the precision-recall curve [AUPRC] and F1). The outcome was 7-year overall survival, evaluated using the AUPRC and Brier scores. Performance was compared against a hospital-only baseline using absolute values and deltas (ΔAUPRC and ΔBrier). Differences and corresponding 95% CIs were estimated on the held-out test set using 2000 bootstrap samples. Zero-shot application reduced the AUPRC in most settings, and any marginal improvements observed in the remaining settings were not statistically significant. In contrast, the domain adaptation model improved AUPRC in 8/12 combinations, with 4 statistically significant gains; the best setting (XGBoost+RUS+F1 optimization) achieved AUPRC=0.5391 (Δ+0.1474; P<.001). The soft ensemble increased AUPRC in 7/12 combinations, with 3 statistically significant gains; the best setting (XGBoost+RUS+AUPRC optimization) achieved AUPRC=0.5060 (Δ+0.1258, P=.002). For calibration, Brier scores improved in most domain adaptation and ensemble combinations, with a substantial proportion reaching statistical significance. When domain adaptation using local hospital data was applied, the model pretrained on synthetic data exhibited similar performance to the hospital-only baseline across various settings. This study demonstrates the methodological utility of a model transfer approach using NCDC synthetic data in a setting with limited data sharing. At the same time, it clarifies that while synthetic data can serve as a complement to local clinical data, it is not a substitute for real-world clinical models.