I have recently finished reading an interesting editorial by Wasserstein et al.¹ on the interpretation and use of p-values in literature and will be sharing my reflection and thoughts on the topic.

As a clinician and researcher, I am well familiar with how heavily p-values are used in biomedical literature. It is not uncommon to see a cut-off threshold of 0.05 being set and p-values below this level interpreted as statistically significant followed by association or causality inferences being made based on this assertion.

As clinicians, it seems natural to look for associations and have the desire to cast away uncertainties. However, the field of statistical inference has various complexities and subtleties which are not always well-taught or well-understood. Although sole reliance on p-values to infer an association of effect or lack thereof has been heavily discouraged, there is often a lack of clarity surrounding alternative or more optimal methods of statistical inference.

The idea of dichotomization of p-values to “significant” and “not significant” is appealing as it seems intuitive and easy to understand. However, this creates an illusion of certainty with our statistical testing when this may not be the case – an unsafe and risky practice. Although we are familiar with the cut-off of 0.05 being arbitrary, we continue to use it as it is “common practice” and well-accepted within the community.  

Uncertainty in statistics is not to be feared and avoided but to be welcomed as it is inevitable. If we embrace the uncertainty, we can then understand how to leverage it and how to understand it better. I found many of the suggestions in this editorial useful as alternatives to the dichotomization of p-values and the idea of statistical significance. Some of these interesting ideas included the use of “second generation p-values” which also considers the impact of the difference found in testing and “false positive risk” which highlights the risk of relying on p-values.

This shift in our thinking and publication practices will take some time but is a necessary process to improve how we best use and interpret our data.

¹Ronald L. Wasserstein, Allen L. Schirm & Nicole A. Lazar (2019) Moving to a World Beyond “p < 0.05”, The American Statistician, 73:sup1, 1-19, DOI: 10.1080/00031305.2019.1583913

This blog post was generated in response to an assignment for the Spring 2023 PQHS-432 course at CWRU.