How do reviewers affect the final outcome? Comparison of the quality of peer review and relative acceptance rates of submitted manuscripts.

Abstract

The purpose of this study was to clarify how peer reviewers affect the relative rate of acceptance of manuscripts submitted to AJR: American Journal of Roentgenology. Manuscript peer reviews for AJR are evaluated and rated by the journal editors on a scale from 1 (lowest) to 4 (highest). These scores are subjective; they are not based on well-defined criteria and are not specifically defined other than as review quality. We obtained all peer review performance scores for the six main types of manuscripts received by AJR as initial submissions (as opposed to revisions) over 5 years and categorized the manuscripts into four groups based on the peer review performance score (not the manuscript rating). Statistical analysis included evaluation of differences in the relative acceptance rates of the manuscripts among the four groups. The relative acceptance rates of manuscripts in the lower review performance score groups (scores 1, 2, and 3) were significantly higher than those of the highest review score group (score 4) for Original Research (p=0.036, p<0.0001, p<0.0001) and Pictorial Essay (all p<0.0001, except for score 3) manuscripts. There was correlation between the quality of peer review performance and the relative acceptance rate of the manuscripts. It is important for AJR to retain highly rated reviewers to maintain its high publishing standards.