Verification is All You Need: Prompting Large Language Models for Zero-Shot Clinical Coding.

Abstract

Clinical coding translates medical information from Electronic Health Records (EHRs) into structured codes such as ICD-10, which are essential for healthcare applications. Advances in deep learning and natural language processing have enabled automatic ICD coding models to achieve notable accuracy metrics on in-domain datasets when adequately trained. However, the scarcity of clinical medical texts and the variability across different datasets pose significant challenges, making it difficult for current state-of-the-art models to ensure robust generalization performance across diverse data distributions. Recent advances in Large Language Models (LLMs), such as GPT-4o, have shown great generalization capabilities across general domains and potential in medical information processing tasks. However, their performance in generating clinical codes remains suboptimal. In this study, we propose a novel ICD coding paradigm based on code verification to leverage the capabilities of LLMs. Instead of directly generating accurate codes from a vast code space, we simplify the task by verifying the code assignment from a given candidate set. Through extensive experiments, we demonstrate that LLMs function more effectively as code verifiers rather than code generators, with GPT-4o achieving the best performance on the CodiEsp dataset under zero-shot settings. Furthermore, our results indicate that LLM-based systems can perform on par with state-of-the-art clinical coding systems while offering superior generalizability across institutions, languages, and ICD versions.