Preparing for AI in Resident Selection: A Scoping Review of Current Applications and Limitations.

Abstract

Residency programs face a large number of applications with limited time to review them. Artificial intelligence (AI) tools could potentially streamline this process. This scoping review examines how AI has been applied in the resident selection process, the impacts described, and the limitations identified. PubMed, OVID, Cochrane, EMBASE, Google Scholar, and Web of Science. A comprehensive search strategy was developed using keywords related to artificial intelligence, natural language processing, machine learning, medical residency, and personnel selection. Studies were included if they were published in English in the last 10 years, involved real or simulated medical or surgical residency applicants or applications, and used AI in any part of the selection process. Articles were identified and screened independently by two reviewers, with conflicts resolved by consensus. Of 115 identified articles, 11 met inclusion criteria. Applications of AI fell into three thematic areas: assessing bias (n = 4), predicting selection outcomes (n = 5), and comparing AI to human reviewers (n = 2). AI models included natural language processing, machine learning, and large language models. While several studies demonstrated AI's potential to detect bias, improve reviewer consistency, and identify overlooked applicants, concerns remain about bias replication, limited generalizability, and the interpretability of AI-driven decisions. Artificial intelligence holds promise for a more efficient and equitable resident selection process. Further investigation is needed to guide responsible development, validation, and oversight of these tools.