There seems to be a reproducibility “crisis” in the sciences, where the results obtained by one group of researchers cannot be reproduced by another, putting the validity of the original work in doubt. What is the fate of those papers? Do they languish in some academic backwater hell lining birdcages?
reproducibility crisis
Much of the literature in the softer sciences, and here we need to include studies of public health issues, like nutrition, exercise, or even COVID-19, seem irreproducible. One group's work does not seem to be easily reproduced by another, giving rise to concerns about bias and veracity. As always, there is far more to the story.
According to pharmacologist Ray Dingledine, good science is hard to do because of (1) "our drive to create a coherent narrative from new data, regardless of its quality or relevance"; (2) "our inclination to seek patterns in data whether they exist or not"; and (3) our negligence to "always consider how likely a result is regardless of its P-value." The good news is that this can be fixed.
The proverbial searching for "the needle in a haystack" can help us understand science's problem with p-values, and why so many studies find contrary things.