Integration of deep learning and radiomic features from multiplex immunohistochemistry images for reproducible Multi-Outcome prediction in a Multi-Center study of colorectal cancer.

Abstract

To develop and validate a robust, multimodal machine learning framework integrating radiomic and deep learning features from multiplex immunohistochemistry (mIHC) images for comprehensive outcome prediction in colorectal cancer (CRC). This multi-institutional retrospective study included 2,117 CRC patients from seven centers, with 1,548 cases used for model training and internal testing, and 569 for external validation. mIHC-stained whole-slide images targeting six immune markers (CD3, CD8, CD45RO, PD-1, LAG-3, Tim-3) were analyzed from two spatial compartments: tumor center and invasive margin. Radiomic features (n = 71/region/marker) were extracted using HistomicsTK, while 768-dimensional deep features were derived using a pre-trained Vision Transformer (ViT-B/16). Feature robustness across biomarkers was quantified via intraclass correlation coefficients (ICC ≥ 0.75). Selected features underwent multi-step selection (LASSO, MI, RFE) and were fused into a single feature space, followed by PCA-based dimensionality reduction. Five clinical tasks were modeled: tumor recurrence, survival status, overall survival duration, TNM staging, and immune profile classification. Classification models (TabTransformer, XGBoost, TabNet) and survival models (DeepSurv, CoxPH, RSF) were trained using 5-fold cross-validation and tested on independent cohorts. Fused features significantly outperformed individual modalities across all tasks. TabTransformer with LASSO-selected fused features achieved top performance: recurrence (AUC = 95.9%), survival status (AUC = 94.5%), TNM staging (macro-AUC = 91.0%), and immune profile (macro-AUC = 91.0%). For survival regression, DeepSurv achieved a C-index of 0.82 and time-dependent AUC of 0.82. Models exhibited strong generalizability, with negligible performance drop on external datasets. SHAP analysis confirmed feature interpretability, with fused features contributing the most across tasks. This study demonstrates that fused mIHC-derived radiomic and deep features yield accurate, interpretable, and generalizable predictions for multiple CRC outcomes, supporting their integration into precision oncology workflows.