In this 2018 commentary, John Ioannidis argues the reporting and analysis of nutritional epidemiology are fundamentally flawed. He writes, “The emerging picture of nutritional epidemiology is difficult to reconcile with good scientific principles. The field needs radical reform.”
Ioannidis reviews some of the dramatic — even implausible — claims that have emerged from epidemiological research including:
- Eating 12 hazelnuts per day will extend lifespan by 12 years; drinking three cups of coffee per day will have the same effect.
- Eating one mandarin orange each day will extend lifespan by five years.
- Consuming one egg daily will shorten lifespan by six years, and two daily strips of bacon will shorten lifespan by a decade. The latter effect is greater than that of smoking.
The authors of such studies generally describe these correlations using causal language, such as “optimal consumption of risk-decreasing foods results in a 56% reduction in all-cause mortality.” Media reports often use causal language even when the authors do not.
Ioannidis argues these effects do not represent the true impact of individual food items but rather the cumulative biases present in this sort of research. Consumption of each food item is correlated, positively or negatively, with hundreds of others, and with an unknown number of nondietary factors that affect health (e.g., geography, education, income, smoking habits). We do not have a sufficient enough understanding of how these factors correlate with one another to correct for their impact, and given that there are over 250,000 foods in the U.S. food supply, fully understanding these correlations may be impossible. As a result, a single correlation drawn from nutritional epidemiology reflects any true effect that food item may have and the various confounders it is correlated with to an unknown extent.
Given this ambiguity and the thousands of possible correlations drawn from any epidemiological data set, it is easy for researchers to favor the correlations most consistent with their existing beliefs about diet and disease. Meta-analyses become “weighted averages of expert opinions” rather than objective assessments of the data. This is exacerbated by the fact that analyses of a single data set can be split across hundreds or thousands of papers; the Nurses’ Health Study alone has yielded more than 1,000 publications. When each association is reported in isolation, it is given exaggerated significance and not seen for what it is: one of thousands of possible associations, each confounded, which could have been drawn from the same data
At minimum, improving the quality of nutritional epidemiology will require publicizing the data from large trials and discarding any past claims in public health, media, or policy drawn from epidemiological studies’ causal claims. Future reform will require greater data transparency, data sharing, and a shift in how this research is presented and analyzed within and beyond the research community.