What is the value of a p value?

Abstract

Successful publication of a research study usually requires a small p value, typically p < 0.05. Many clinicians believe that a p value represents the probability that the null hypothesis is true, so that a small p value means the null hypothesis must be false. In fact, the p value provides very weak evidence against the null hypothesis, and the probability that the null hypothesis is true is usually much greater than the p value would suggest. Moreover, even considering "the probability that the null hypothesis is true" is not possible with the usual statistical setup and requires a different (Bayesian) statistical approach. We describe the Bayesian approach using a well-established diagnostic testing analogy. Then, as a practical example, we compare the p-value result of a study of aprotinin-associated operative mortality with the more illuminative interpretation of the same study data using a Bayesian approach.