Urological trials without significant results. Can we extract meaning from disappointment?

Abstract

To calculate additional information from urological trials which do not report statistically significant results, and assess whether language manipulation is used when such studies were reported. Many randomised controlled trials (RCTs) do not yield statistically significant primary outcomes; such results are challenging to interpret using traditional statistical practices. An alternative approach, the likelihood ratio (LR), uses Bayesian statistical methods to compare the null (no significant effect of treatment) and alternative (significant effect of treatment) hypotheses, and provides quantitative strength of evidence for one compared to the other. A lower LR indicates stronger support for the alternate hypothesis over the null and vice versa. We performed a cross-sectional review of urological RCTs, published in the period 2018-2024, with a primary outcome that did not reach statistical significance. For each outcome, an LR was calculated. For each included article the use of language manipulation ('spin') was also analysed. Eighty-two articles with 98 primary outcomes were identified. For 13% of these outcomes the data provided greater support for the alternative hypothesis compared to the null, as indicated by an LR < 1. This suggests these trials, such as the EVEREST trial (LR = 0.26), should be considered for further investigation. For 36% of results the LR was >100, indicating decisive evidence in favour of the null hypothesis, and suggesting these trials, including SWOG S1011 (LR = 250), need not be repeated. Spin was identified in 41.5% of abstracts. After adjustment, only impact factor and LR <1 were associated with spin use. This study was limited by its analysis of only articles published in high impact factor journals in the past 7 years. Many urological RCTs with primary outcomes that were not statistically significant by traditional measures warrant further investigation. Researchers should apply the LR to future trials to aid interpretation of their results.