Benchmarking large language models for cell typing in single-cell RNA-Seq.

Abstract

Large language models (LLMs) show significant potential for cell type annotation in single-cell RNA sequencing (scRNA-seq), but a systematic framework for their application and a comprehensive performance comparison are lacking. Here, we systematically benchmarked seven leading LLM models and three traditional bioinformatics tools across 34 diverse human and mouse datasets. We establish that using marker genes selected by statistical significance and ranked by log₂ fold change optimizes annotation accuracy. Our benchmark reveals that LLMs profoundly outperform traditional methods, particularly in resolving fine-grained cell subtypes. A top tier of models, including Kimi-k2, GPT-5, Claude-4.1, and Grok-4, consistently delivered the highest accuracy. To harness their collective strength, we developed an elite ensemble strategy that achieves state-of-the-art performance. We encapsulated these findings into DeepCellSeek, an open-source R package and interactive web platform, to provide a validated, high-performance solution. This work provides a practical roadmap for leveraging LLMs in single-cell research and paves the way for their evolution into powerful discovery engines.