Null studies in ophthalmology journals: a large-scale analysis of PubMed indexed research.

Abstract

Null studies, in which the primary outcome is not statistically significant, are essential for unbiased evidence-based medicine and help reveal selective dissemination of research. We estimated the prevalence of null studies and characterized differences between null and positive studies in ophthalmology journals. Retrospective analysis of original studies published from January 1, 2014, through December 31, 2025. Original studies published in 90 PubMed-indexed journals listed in the Ophthalmology category of the 2025 Journal Citation Reports. Study characteristics were derived using rule-based processing, except classification of the main study result as null versus positive, which used a large language model pipeline (gpt-4.1) applied to abstracts. Large language model performance was evaluated against a prespecified manual validation set of 500 abstracts, achieving kappa 0.87 (prespecified threshold 0.75). Prevalence and differences in study characteristics between null and positive studies. A total of 93,963 studies were included, of which 19,231 (20.5%) were null studies. The proportion of null studies declined from 25.6% in 2014 to 15.1% in 2025 (p <0.001). Positive studies were more often funded than null studies (68.4% vs 58.0%; risk difference 95% CI 9.6 to 11.2; p<0.001) and were more often prospective in design (17.5% vs 16.0%; risk difference 95% CI 0.9 to 2.0; p<0.001), whereas randomized design was similarly common between groups (6.5% vs 6.5%, p=0.867). Positive studies were also more often drug-focused (13.6% vs 11.8%), device-focused (9.3% vs 6.9%), or surgery-focused (13.4% vs 9.3%) (all p<0.001). Null studies were published in journals with lower mean impact factor (2.59±1.62 vs 2.73±1.69; mean difference 95% CI 0.11 to 0.17; p<0.001), and worse mean journal rank (40.11 ± 22.6 vs 37.80 ± 22.2; mean difference 95% CI -2.67 to -1.95; p<0.001) compared to positive studies. In this large-scale analysis of ophthalmology journals, one in five studies had a null primary outcome. Overall, null studies became less common over time and were more often published in lower-impact, lower-ranked journals and were more likely to be unfunded. These findings suggest systematic differences in the characteristics and dissemination of null versus positive studies in ophthalmology.