Cross-linguistic benchmarking of NLP metrics for psychosis research.
Where this comes from
- Record sourced from PubMed, PMID 42547525.
- Also identified by DOI 10.1038/s41746-026-03053-y and PMC identifier 13433947.
- Licence recorded as CC BY.
- The licence permits redistribution, so the abstract is shown in full and the full text is available from the publisher.
Abstract
Language is central to psychosis, making it a key focus for both clinical evaluation and research. Many automated linguistic metrics have been developed to analyze speech in psychosis, yet few studies have compared them systematically. In this study, we benchmarked the performance and robustness of 51 natural language processing (NLP) metrics using two datasets: one with German- and one with Russian-speaking participants. Metric performance was defined as the metrics' capacity to reflect symptom severity and distinguish individuals with psychosis from healthy controls. The best-performing metrics tended to be robust across different clinical rating scales and speech elicitation tasks. Overall, verbosity metrics (e.g., word count) outperformed most other metrics across tasks and languages. Thus, verbosity-based metrics should be used as a baseline when evaluating cross-linguistic NLP metrics that are both clinically meaningful and useful for psychosis research.