Safety and security of large language models in healthcare.

Abstract

Integration of artificial intelligence methods into clinical care is proceeding rapidly, driven by advances in generative artificial intelligence, most notably large language models. Large language models trained on large amounts of text have shown potential across nearly every domain of healthcare. However, their broad applicability also comes with new responsibilities, vulnerabilities and threats. These need to be assessed and mitigated before widespread clinical adoption. Here we review the available literature on security and safety of large language models themselves as well as their integration with hospital workflows and interactions with human healthcare providers. We systematically map security hazards to development stages of clinical artificial intelligence systems (design, data, model, inference and environment), identify safety layers, from core optimization objectives, knowledge integrity and alignment, to interaction with humans and systems, and classify threats by their current clinical relevance. Finally, we provide a perspective on current mitigation techniques, illustrating respective stakeholders' responsibilities.