Public Board Score Reporting Undermines Holistic Review for Residency Selection.

Abstract

Holistic review has become the gold standard for residency selection. As a result, many programs are de-emphasizing standardized exam scores and other normative metrics. However, if standardized exam scores predict passing of an initial certifying exam, this may lead to an increase in board failure rates within specific residency training programs who do not emphasize test scores on entry. Currently, the board pass rates of residency programs from many of the American Board of Medical Subspecialities (ABMS) are publicly reported as a rolling average. In theory, this should create accountability but may also create pressure and distort the way residency program selects applicants. The risk to programs of having a lower board pass rate publicly reported incentivizes programs to focus increasingly on standardized test scores, threatening holistic review. All programs do not recruit students entering residency with an identical chance of passing boards. Therefore, we believe the ABMS member boards should stop publicly reporting raw certifying exam rates above a certain threshold for normative comparison. We strongly encourage the use of learning analytics to create a residency "expected board pass rate" that would be a better metric for program evaluation and accreditation.