Keys' Scientific Abandon

Ancel Keys launched his landmark Seven Countries Study (1) in 1958, pioneering both the use of huge population studies to establish relationships between food and health and an influential understanding of the nature of a healthy diet that has been with us ever since. The Seven Countries Study was essential in forming the belief that consuming saturated fat increases blood cholesterol, leading in turn to atherosclerosis and heart disease. It was one of the largest and most ambitious scientific projects undertaken at the time, and Keys quickly became a well-known public authority on heart disease.

Certain critical issues, however, have always undermined the study’s core conclusions. They are manifest in Keys’ thinking and have been institutionalized in nutritional epidemiology research ever since. To understand these issues, it helps to look at Keys’ earliest work on this “diet-heart hypothesis” and how it evolved.

Keys formulated his diet-heart hypothesis based on research he carried out personally in Greece, Italy, Spain, South Africa, Japan, and Finland. This led to a 1953 publication laying out his thinking and supporting his conclusions with what he would later call a “remarkable relationship” between fat consumption and heart-disease fatality visible even in easily available national statistics. While Keys’ 1953 argument and the supporting evidence were simplistic, the problems with its logic are still characteristic of much of nutrition science today.

Figure 1: Graphing of all 22 countries shows a weaker correlation than what Keys presented. (CrossFit Inc.)

In his 1953 article “Atherosclerosis: A Problem in Newer Public Health,” Keys published a graph (Figure 2 below) linking fat-calorie availability in 1949 in six countries to deaths from coronary heart disease in middle-aged men. The graph seemed to demonstrate a striking, near-perfect correlation between fat intake and degenerative-heart-disease mortality, ranging from fewer than 1 in 1,000 deaths in the under-10-percent-fat-eating Japanese to over 7 in 1,000 deaths from the nearly 40 percent fat diets of Americans, with Italy, England and Wales, Australia, and Canada falling neatly along the curve between them (2).

Figure 2: Keys’ 1953 graph, presented as follows: “Mortality from degenerative heart disease (categories 93 and 94 in the Revision of 1938, categories 420 and 422 in the Revision of 1948, International List. National vital statistics from official sources. Fat calories as percentage of total calories calculated from national food balance data for 1949 supplied by the Nutrition Division, Food and Agriculture Organization of the United Nations).”

The graph caused an international stir, thrusting Keys’ favored diet-heart hypothesis—that dietary fat increases cholesterol in the blood and elevated cholesterol causes heart disease, hence dietary fat causes heart disease—to the fore and launching its trajectory from hypothesis to “proven” fact, where it has persisted for half a century.

In “Fat in the Diet and Mortality From Heart Disease; A Methodologic Note,” Jacob Yerushalmy and Herman E. Hilleboe, both experienced statisticians in the public-health world, exposed the problems with Keys’ methodology and statistical analysis as well as his conclusion that the data supported a causative role for fat as the driver in the development of degenerative heart disease (3).

Based on the data expressed in the graph in his 1953 paper, Keys had speculated that “dietary fat somehow is associated with cardiac disease mortality at least in middle age.” By the time he got to the Conclusions section in the same paper, his speculation had strengthened to this: “The different age-specific death rates of men 40 to 65 from degenerative heart disease in different countries are directly related to the differences in those countries in the proportion of the total calories derived from total fats.”

While this bolder statement could be drawn from the data Keys presented for the six countries he selected for his graph, there were several methodological and analytical flaws, as Yerushalmy and Hilleboe pointed out in their 1957 rebuttal (and in a separate 1957 paper published by Hilleboe), beginning with the selection bias inherent in the data. This was the problem with Keys’ basing his selection on his exploratory research—the scientific equivalent of stacking the deck in a game of cards:

“In studies of association, as in any other study, the method of selecting data largely determines whether the results can safely be generalized. … Hence, it is of first importance to know on what basis the six countries were selected, in order to determine whether the findings for them can be generalized to other countries. Since no information is given by Keys on how or why the six countries were selected for (his graph), it is necessary to investigate the association between dietary fat and heart disease mortality in all countries for which information is available.”

If data of 12 countries had been included in the graphs, as shown below in Hilleboe’s figure, the case for an association (let alone a causative relationship) between dietary fat consumption and death from degenerative heart disease becomes less compelling (4).

Figure 3: Hilleboe’s graphs, presented as follows: “Scatter diagrams of 12 countries according to percent of calories derived from fat and mortality from heart disease for males, by age.”

Yerushalmy and Hilleboe further noted that relevant information was available for 22 countries at that time and went on to describe the dangers of selective parsing of the data, such as the method employed by Keys:

“As an illustration of the importance of this question of selection, it may be of interest to show how by selection of countries it is possible to ‘demonstrate’ an apparent inverse relationship between percent of calories from fat and mortality from vascular lesions affecting the central nervous system (i.e., stroke).” 

They used Figure 4 below to illustrate the point.

Figure 4: Yerushalmy and Hilleboe’s graph, presented as follows: “Mortality from vascular lesions affecting the central nervous system (B-22) and fat calories as per cent of total calories in males fifty-five to fifty-nine years from six countries selected for this specific purpose. Calculated from national food balance data by F.A.O. (Food and Agriculture Organization of the United Nations).”

Keys’ interpretation of this data and failure to understand basic principles of scientific methodology were then manifest in the subsequent Seven Countries Study, in which he once again selected his countries and the sites of study knowing in advance what he was likely to find (5). That more famous work would be criticized by many, including Russell Smith, Ph.D. A psychologist with a strong background in mathematics and physiology, Smith concluded:

“The word ‘landmark’ has often been used to describe Ancel Keys’ Seven Countries Study, commonly cited as proof that the American diet is atherogenic. The dietary assessment methodology was highly inconsistent across cohorts and thoroughly suspect. In addition, careful examination of the death rates and associations between diet and death rates reveal a massive set of inconsistencies and contradictions. . . . It is almost inconceivable that the Seven Countries Study was performed with such scientific abandon.” (6)

Whether or not there truly was, or is, a correlation between fat intake and heart disease in the countries Keys surveyed, the data was insufficient to clearly show fat intake was the primary factor affecting heart-disease risk. Unfortunately, these biased data and the ways they were interpreted by Keys and others helped cement the belief that dietary fat causes heart attacks, and so the diet-heart hypothesis.

References

1. Keys A. Coronary heart disease in seven countries. Circulation 41(1): 186-195, 1970.

Keys and his team published the results of the Seven Countries Study across a number of publications, including multiple book-length summaries. A full list of publications can be found here. 

2. Keys A. Atherosclerosis: a problem in newer public health. Journal of the Mount Sinai Hospital, New York 20(2): 118-139, 1953. Available here.

Keys’ review of the clinical significance of atherosclerosis and the significance of cholesterol in heart-disease risk (and of diet to blood cholesterol levels). Keys begins by noting high levels of heart-disease mortality are not inevitable given significantly lower rates observed in other nations (citing Italy as a specifically significant example). He also notes that heart-attack patients tend to have higher total cholesterol levels (as do diabetics) than the general population, concluding “measurement of cholesterol and allied substances in the blood serum affords an indication of the tendency towards the development of atherosclerosis and degenerative heart disease.” Regarding the effect of diet on heart disease, he cites observations of a significant decrease in serum cholesterol on “a modified rice-fruit diet” containing “no cholesterol and extremely little fat,” as well as, more famously, a graph showing a positive relationship between fat intake (as a percentage of total calories) and heart-disease mortality across six countries (Japan, Italy, England/Wales, Australia, Canada and the U.S.). Keys notes: “All the data summarized here suggest an important chain of relations between the total fat content of the diet (or the proportion of fat calories of the total metabolized), the cholesterol (and lipoprotein) concentration in the blood, the development of atherosclerosis, and the mortality from degenerative heart disease. … It must be concluded that dietary fat is somehow associated with cardiac disease mortality, at least in middle age.” He suggests the high levels of fat intake in the U.S. may be related to the high levels of cardiovascular mortality seen in the same.

3. Yerushalmy J and Hilleboe HE. Fat in the diet and mortality from heart disease; a methodologic note. New York State Journal of Medicine 57(14): 2,343-2,354, 1957. Available here.

Yerushalmy and Hilleboe drew on data from the Food and Agriculture Organization of the United Nations (FAO) to reassess Keys’ famous 1953 conclusion that share of calories from fat and coronary mortality rates were strongly correlated. Analyzing data from 22 countries, they found a weaker (but still significant) correlation between dietary fat consumption and heart disease than Keys. They also found animal protein correlated more strongly with coronary heart-disease mortality than proportion of calories from fat. Perhaps most importantly, they noted fat intake may correlate with heart-disease mortality as a proxy of a country’s development, and so not be the driver of heart disease. Note, however, that by the time this study emerged, Keys had already begun preparations for the Seven Countries Study, and so this analysis may have had a limited impact on it.

4. Hilleboe HE. Some epidemiologic aspects of coronary artery disease. Journal of Chronic Diseases 6(3): 210-228, 1957.

Hilleboe reviews different factors correlating with heart disease, concluding an association exists between calorie and/or fat intake and heart disease, but that the existing information can “be used to form hypotheses for field testing.” Further: “It is apparent from these scatter diagrams that there is no sound basis for the statement that a strong parallelism in the different countries exists between fat consumption and heart disease mortality. In countries where the calories derived from fat were between 30 and 40 per cent, there were extreme variations in the cardiac death rates, in each of the seven age groups shown.”

5.  Sevencountriesstudy.com. Countries and cohorts. (n.d.). Available here. 

6.  Smith RL and Pinckney ER. Diet, Blood Cholesterol, and Coronary Heart Disease: A Critical Review of the Literature (Vol. 2). Sherman Oaks, California: Vector Enterprises, 1991. Available here.

All links accessed Dec. 31, 2018.