Estimating the Causal Effect of Double-Blind Peer Review for a Pulmonary, Critical Care, and Sleep Medicine Journal.

Abstract

Double-blind peer review (DBPR; reviewers masked to author identities and vice versa) aims to reduce biases. The effectiveness of DBPR has not been studied in pulmonary, critical care, and sleep journals. What was the causal effect on bias of DBPR at CHEST? From January 2020 to June 2022, CHEST and Annals of the American Thoracic Society (AATS) used single-blind peer review (reviewers concealed from authors only); in July 2022, CHEST switched to DBPR. We estimated the causal effect of DBPR on manuscript acceptance at CHEST using AATS as a control. The co-primary exposures were first and last author gender; author residence in English-fluent countries was considered as a secondary exposure. To estimate DBPR's differential impact according to exposure, multivariable regression models were constructed with triple interaction terms (eg, author gender × CHEST vs AATS × pre- vs post-DBPR), all component factors (eg, author gender), paired interactions (eg, author gender × CHEST vs AATS), and covariables (year, topic). The study included 4,651 manuscripts (CHEST, 3,494; AATS, 1,157) sent for peer review; 4,645 (99.9%) had identifiable author gender. From 2020 to 2024, CHEST submissions (compared with AATS) were less frequently authored by women (first, 38.7% vs 45.5% [standardized mean difference (SMD), 0.14]; last, 23.9% vs 31.2% [SMD, 0.16]) or people from English-fluent countries (first, 56.0% vs 77.7% [SMD, 0.47]; last, 57.9% vs 79.0% [SMD, 0.47]), and acceptance rates were lower (30.7% vs 48.4%; SMD, 0.37). Following adjustment, no difference was appreciated in the impact of DBPR on acceptance for women vs men authors (first, -7.4 [95% CI, -20.9 to 6.0] percentage points [P = .28]; last, -10.1 [95% CI, -25.1 to 5.0] percentage points [P = .19]). DBPR negatively affected first (but not last) authors from English-fluent countries (vs non-fluent, -16.8 [95% CI, -32.1 to -1.6] percentage points; P = .030). Our results show that DBPR did not differentially affect authors by gender but did by presumed English fluency.