Demonstrating Construct Validity of the American Board of Physical Medicine and Rehabilitation Part I Examination: An Analysis of Dimensionality.

Abstract

Ideally, high-stakes examinations assess 1 dimension of medical knowledge to produce precise estimates of a candidate's performance. It has not been reported whether the American Board of Physical Medicine and Rehabilitation Part 1 Certification Examination (ABPMR-CE-1) is unidimensional or not. To examine the ABPMR-CE-1 to measure how many dimensions it assesses. Retrospective observational study. We assessed examination results from the 2015 ABPMR-CE-1. A total of 489 deidentified candidates taking the 2015 ABPMR-CE-1. A 1-parameter Item Response Theory (IRT) measurement model was utilized. A Principal Components Analysis (PCA) of standardized residual correlations was used to detect multidimensionality. Number of primary dimensions reflected in the 325 test questions. The results of the dimensionality analysis indicated that the ABPMR-CE-1 examination is highly unidimensional from a psychometric perspective. Expert content review of the substantive content of small contrasting clusters of questions provided additional assurance of the unidimensional nature of the examination. The ABPMR-CE-1 appears indeed to measure a single construct, which suggests a sound structure of the examination. It closely approximates the assumption of statistical unidimensionality. Not applicable.