Advancing medical AI through benchmarking and competition for specialty triage.

Abstract

Artificial intelligence holds transformative potential for clinical triage, yet challenges in accuracy, generalization, and interpretability persist. To address these gaps, we introduce MedTriage, a benchmark designed to evaluate large-scale models across diverse clinical scenarios rigorously. Leveraging this framework, we launched the Large-Model-Based Medical Triage Evaluation Competition, utilizing real-world clinician-patient dialogues from general hospitals and four specialized domains. The competition engaged numerous research teams, spurring advancements in large-model-driven triage algorithms. Building on the competition insights, we developed an enhanced model (MedGPT-Guide) employing a "10 Relevant + 10 Random + Ensemble" strategy, achieving superior accuracy on the MedTriage benchmark. Our results underscore the power of "evaluation-driven training" to improve model performance and lay the groundwork for standardized, deployable intelligent triage systems. Moving forward, priorities include enhancing data security, model generalization, and addressing legal and regulatory frameworks.