John Ioannidis is like the Fourth Horseman of the Apocalypse. When he comes riding in, scientists tremble in fear.
This is for good reason. He first burst onto the national scene in 2005 with a groundbreaking paper titled simply "Why Most Published Research Findings Are False." His statistical analysis and logic are impeccable, and his paper has never been seriously refuted. Furthermore, he has had a tremendous impact: The paper has been viewed more than 2.5 million times, which is the scientific equivalent of a viral Katy Perry video.
Since then, Dr. Ioannidis has gone on to show that the best scientists don't always get funded, why neuroscience is unreliable, why most clinical research is useless, and that most economics studies are exaggerated. In other words, the process by which we acquire new knowledge is fundamentally flawed and much of what we think we know is wrong. Dr. Ioannidis is not just a bull in a china shop; he's a bazooka in a china shop.
And now the bazooka is aimed at nutrition research.
Nutrition Research Needs 'Radical Reform'
Here at ACSH, we have been saying for a long time that nutrition research is shoddy and mostly wrong. The reason is inherent to the way research is conducted in the field: Too much of it relies on food frequency questionnaires (FFQs), which ask people what and how much they ate. Humans are notoriously bad at remembering things like this, which is why research has linked pretty much everything to cancer. We also lie about stuff, like how much alcohol we drink.
In a new op-ed in JAMA, Dr. Ioannidis bluntly states that nutrition epidemiology is in need of "radical reform." In a paragraph that perfectly captures the absurdity of the field, he writes:
"...eating 12 hazelnuts daily (1 oz) would prolong life by 12 years (ie, 1 year per hazelnut), drinking 3 cups of coffee daily would achieve a similar gain of 12 extra years, and eating a single mandarin orange daily (80 g) would add 5 years of life. Conversely, consuming 1 egg daily would reduce life expectancy by 6 years, and eating 2 slices of bacon (30 g) daily would shorten life by a decade, an effect worse than smoking. Could these results possibly be true?"
The answer to his rhetorical question is obviously no. So, how did this garbage get published?
Dr. Ioannidis blames residual confounding and selective reporting. Confounding means incorrectly concluding that A causes B when, in reality, some other factor X causes B. The trouble for researchers is when A and X are related to each other. Teasing out the true cause can be quite difficult. For instance, eating bacon very well may be associated with a shorter lifespan. But maybe bacon-eaters are also less likely to exercise, and lack of exercise (the confounding factor) is the actual cause of a shorter lifespan.
Residual confounding refers to the undeniable fact that we can't know if confounding is present if we don't even bother to measure confounders in the first place. (And, to complicate matters, it may not even be possible to measure some potential confounders, such as how a person's lifestyle might change over time, what his genetic background is, or the exact chemical composition of the thousands of different foods in the marketplace.)
Selective reporting means that any study which shows a link between bacon and early death is likelier to be published than one that doesn't show a link. Combined, Dr. Ioannidis believes that residual confounding and selective reporting have created a systemic bias in nutrition research.
Even worse, Dr. Ioannidis blames food and nutrition activists for distorting research. He says, "[I]nstead of carefully conducted primary studies informing guidelines, expert-driven guidelines shaped by advocates dictate what primary studies should report." In other words, opinions drive what data are collected and reported, rather than data driving what opinions researchers hold. That is precisely backwards. It's not a surprise, therefore, that when tested in clinical trials, results from nutritional epidemiology often fail to replicate.
How Can Nutrition Research Be Fixed?
The solutions that Dr. Ioannidis suggests aren't unique to nutrition: More access to complete data sets, more independent investigations, and more transparency. Unfortunately, the bad habits of this field are so deeply ingrained -- and the special interests who make money by peddling dieting books are so dedicated to maintaining the status quo -- that reform seems unlikely.
Source: John P. A. Ioannidis. "The Challenge of Reforming Nutritional Epidemiologic Research" JAMA. Published online: August 23, 2018. DOI: 10.1001/jama.2018.11025.
