Should LLMs be over empowered for high-stake regulatory research?

Abstract

This letter critically evaluates the feasibility of implementing open-source large language models in regulatory research, building upon the recent study on zero-shot and few-shot learning approaches for regulatory tasks. While the study demonstrates that models like Flan-T5 can effectively extract pharmacokinetic drug-drug interactions and intrinsic factors from Food and Drug Administration (US) drug labels with high precision, it also highlights significant challenges, including computational constraints, performance variability, prompt sensitivity, and the risk of misclassification. To address these issues, this letter discusses intuitive ways for mitigating these limitations.