Correspondence: Accuracy Is Not Enough: Stability-Aware Feature Selection for Reproducible Biomarker Discovery.

Abstract

Random forest (RF) models can achieve high predictive accuracy, yet their model-specific feature importances may be unstable and misleading. Using an allergy benchmark dataset (10,000 instances, 11 features), we compared five selection strategies-RF, logistic regression, feature agglomeration (FA), highly variable gene selection (HVGS), and Spearman correlation-evaluating cross-validated accuracy with the top five features and after removing the top two (reselecting the top three). RF attained 0.9999 accuracy with the top five but fell to 0.8836 and showed unstable rankings; logistic regression maintained 0.9116 but was also unstable. FA, HVGS, and Spearman achieved near-perfect accuracy (0.9999) with the top five and modest declines (0.9076-0.9116) with stable rankings. Results underscore that accuracy does not imply reliable importance; stability-aware, model-agnostic, or unsupervised methods better support reproducible biomarker discovery.