The Framingham Heart Study, Part 3: Framingham’s Presentational Flaws — Bias or Fraud?

For all its landmark status, the Framingham Heart Study (FHS) was riddled with flaws and inconsistencies in methodology and analysis nearly from its inception. Yet perhaps the most damaging sleight of hand occurred in how the material was presented to the public.

Russell Smith, in his aforementioned treatise “Diet, Blood Cholesterol, and Coronary Heart Disease: A Clinical Review of the Literature,” writes: “Framingham investigators seemed to be objective in their early reports but subsequently exhibited stronger and stronger biases. The use of such terms as ‘powerful’ to describe an exceedingly weak relationship has been commonplace. One kind of bias exhibited was the process of reinterpreting data in order to be consistent with the current position of the alliance (the group providing the funding; i.e., the NHLBI).” (9)

It has become apparent that the bias of the National Heart, Lung and Blood Institute (NHLBI) is that the lipid hypothesis—the idea that cholesterol is the primary driving force behind the development of cardiovascular disease—is a proven fact. To that body, it is not just a hypothesis (and a weak one at best) but hard fact. This strong belief in the lipid hypothesis demands a lowering of cholesterol levels by any means necessary, which usually implies the use of drugs. Consequently, the NHLBI is now arguably the largest pusher of statins on the planet. But we get ahead of ourselves.

In an effort to make the lipid hypothesis more plausible, the NHLBI has resorted to what some consider outright fraud (5,10).

Interpretational Errors

If you study the actual data from the FHS, as I have, you will discover that the association of cholesterol with heart disease is a weak one and holds only for men under the age of 50. Above 50, lower cholesterol levels correlate with increased risk of death—each 1 mg/dL drop per year of cholesterol levels equates to an 11 percent increase in both the overall death rate and the cardiovascular-disease death rate. If cholesterol truly is the driving force behind cardiovascular disease, why would a falling blood-cholesterol level drive an increased rate of death from the very disease it is supposed to cause? That question remains unanswered (11).

Shortly after the publication of the paper showing these statistics, the American Heart Association and the NHLBI published a joint statement summarizing the evidence relating dietary fats, serum cholesterol, and coronary heart disease.

This summary contained the famous statement based on the data from the FHS: “A 1% reduction in an individual’s total serum cholesterol level translates into an approximate 2% reduction in CHD (coronary heart disease) risk.” (12)

The citation for this claim was the above-mentioned paper that showed a relationship between a falling serum-cholesterol level and an increase in the death rate for CHD (11). To be fair, that paper did show an increased risk for CHD, as cholesterol went up in those people under 50. But how could the authors be so definitive in their blanket statement that a 1 percent drop in cholesterol brings about a 2 percent reduction in CHD in all subjects when the reference paper clearly describes just the opposite in subjects over 50?

The short answer: They couldn’t. Yet they did.

Below are a few more examples of misdirection on the part of the NHLBI in regard to the FHS.

Framingham and the Diet-Heart Hypothesis

Early on, the FHS investigators meticulously recorded the dietary habits of the subjects. Influenced by Ancel Keys and Jeremiah Stamler (both proponents of the diet/heart hypothesis that held dietary fat as a factor in the development of cardiovascular disease), the NHLBI wanted to see if there were correlations among fat in the diets of the FHS subjects, their subsequent cholesterol readings, and any new CVD.

According to George Mann, here’s how it turned out:

During the years 1955-60, at the Framingham study, a prospective investigation of the nature of CHD, we undertook an extensive study of the relationship of dietary habits to the occurrence of CHD in adult men and women, then 36-64 years old. Detailed dietary studies were recorded on a subsample of about 1,000 subjects by a carefully tested procedure. Miss Georgeana Pearson spent about 3 years interviewing those people. It was a meticulous enterprise. The data clearly showed no relation between dietary intake of either fat or cholesterol and the subjects’ level of cholesteremia (cholesterol in the blood) or their experience of CHD. Those data were never properly published because the findings were contrary to the position held by the NHLBI. (5,10)

More about that later.

Interestingly, Dr. William Castelli echoed those findings in an editorial he wrote in the Archives of Internal Medicine in 1992, while he was director of the FHS.

“For example, in Framingham, Mass., the more saturated fat one ate, the more cholesterol one ate, the more calories one ate, the lower the person’s serum cholesterol.” (13)

Compare this to what he told a New York Times reporter a mere two years later in 1994:

“We now believe it’s the newest lipid deposits—where that greasy cheeseburger you just ate landed—that rupture and precipitate the majority of heart attacks. These deposits don’t impinge on blood flow while on artery walls, but they are unstable and easily rupture to block a coronary vessel. The good news is that these young, unstable deposits can be shrunk. Even if you have a heart attack, after two years of lowering your cholesterol, you can dramatically reduce your risk of another attack.” (4)

Where is the evidence for this in the data generated by the FHS? It doesn’t exist.

“He (Castelli) laments that Americans are a long way from ‘aggressive lipid management,’ which would enable 75 percent of them to lower their cholesterol through diet and exercise alone to a level where a heart attack is highly unlikely. The remaining 25 percent would need the help of cholesterol-lowering drugs, he said, adding that only 5 percent are currently on such medication” (my italics) (4).

The New York Times article from which these quotes were taken goes to great lengths to discuss Castelli’s teachings about avoiding fat—especially saturated fat—and dietary cholesterol to reduce cholesterol levels in the blood. How can he square that with his own Framingham study, which he says shows that the more saturated fat, cholesterol, and calories the subjects ate, the lower their cholesterol levels fell?

Was he admonished by his superiors at the NHBLI?

This flip-flopping on the data seems to be a common strategy among directors of the FHS.

As quoted above, Mann wrote that the Framingham dietary “data were never properly published.” He’s correct in that they were never published in a medical/scientific journal for all to read, but they were finally published in 1970.

Tavia Gordon, the head statistician for Framingham at the time, took it upon himself to publish the findings, not in a journal but in a small, privately printed monograph of some 80 pages of reproduced typewriter script bound in a light-blue cover (14).

I was able to track down a copy of this document, and it pretty well adheres to what Castelli said: No real correlations with diet and blood cholesterol and CHD.

What was interesting, though, is an Oct. 30, 1970, editorial from the Framingham newspaper taped to the back page of the document. The editorial extensively quotes Dr. William Kannel, who was the director of the FHS at that time. The document mentioned above had been released, and a reporter had asked for Kannel’s commentary on it.

Here is the opening paragraph of the editorial titled “Findings of Framingham Diet Study Clarified” (15):

“Although there is no discernible relationship between reported diet intake and serum cholesterol levels in the Framingham Diet Study group, ‘it is incorrect to interpret this finding to mean that diet has no connection with blood cholesterol,’ Dr. William B. Kannel, director of the Framingham Heart Study, has stated.”

The editorial goes on to further clarify Kannel’s views on cholesterol.

“A number of blood lipids have been implicated in coronary artery disease, Dr. Kannel said, but none more substantially than the blood-cholesterol content. That blood cholesterol is somehow intimately related to coronary atherosclerosis is no longer subject to reasonable doubt because of massive evidence that”—the editorial goes on to detail a long list of observational studies upon which Kannel bases his opinion.

“‘Moreover,’ Dr. Kannel said, ‘the evidence incriminating diet in producing elevated serum cholesterol values is quite substantial’”; this last claim is followed by yet another list of observational studies Kannel relies on for his understanding of diet and serum cholesterol.

Because we now all know the value of observational studies, we can see that Kannel’s ideas are built on shifting sands.

I find it remarkable that Kannel can disregard the data from his own study because it doesn’t conform to what he already believes. He was involved in the collection of this data and observed how meticulously the work was done, yet he rejects it.

How can he expect the FHS to have any credibility at all when he rejects this data and, at the same time, fully embraces other data that better fit his views?

Clearly, from the fumbling of data, the misrepresentation of results, the shoddy experimental design, the statistical manipulations, and all the other antics discussed above, the FHS certainly fits Feynman’s categorization of cargo-cult science. More is the pity that it has adversely affected so many lives.

In light of this review, you might be interested to see where you fall on the Framingham Risk Score.

After you’ve calculated your score—based on data so flawed as to be a meaningless predictor—reflect on how many unfortunate people have turned to expensive and not entirely benign drugs and an ineffective, unappealing, and even harmful low-fat, low-cholesterol, low-meat diet as a consequence of scores they received on this same calculator. This is the enduring legacy of the Framingham Heart Study.

---

References

Note: These references include those previously published in “The Framingham Heart Study, Part 1” and “Part 2.”

1. Bruenn HG. Clinical notes on the illness and death of President Franklin D. Roosevelt. Annals of Internal Medicine 72(4): 579-591, 1970. Available here.
2. Dawber TR. The Framingham Study: The Epidemiology of Atherosclerotic Disease. Cambridge, Mass.: Harvard University Press, 1980.
3. Mahmood SS, Levy D, Vasan R et al. The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective. The Lancet 383(9921): 999-1008, 2014. Available here.
4. Brody JE. Scientist at work: William Castelli; preaching the gospel of healthy hearts. The New York Times. Feb. 8, 1994. Available here.
5. Mann GV. Coronary Heart Disease: The Dietary Sense and Nonsense. Cambridge, England: Janus Publishing Company, 1993.
6. Feynman RP. Cargo cult science. Engineering and Science 37(7): 10-13, 1974. Available here.
7. Feinstein AR. Scientific standards in epidemiologic studies of the menace of daily life. Science 242(4883): 1257-1263, 1988. Available here.
8. Werkö L. Risk factors and coronary heart disease—facts or fancy? American Heart Journal 91(1): 87-98, 1976. Available here.
9. Smith RL. Diet, Blood Cholesterol, and Coronary Heart Disease: A Critical Review of the Evidence. Sherman Oaks, Calif.: Vector Enterprises, 1991. Vol. 2 available here.
10. Mann GV. Diet-heart: end of an era. New England Journal of Medicine 297(12): 644-650, 1977. Available here.
11. Anderson KM, Castelli WP, Levy D. Cholesterol and mortality: 30 years of follow-up from the Framingham study. Journal of the American Medical Association 257(16): 2176-2180, 1987. Available here.
12. LaRosa JC, Hunninghake D, Bush D et al. The cholesterol facts. A summary of the evidence relating dietary fats, serum cholesterol, and coronary heart disease. A joint statement by the American Heart Association and the National Heart, Lung, and Blood Institute. The Task Force on Cholesterol Issues, American Heart Association. Circulation 81(5): 1721-1733, 1990. Available here.
13. Castelli WP. Concerning the possibility of a nut. Archives of Internal Medicine 152(7): 1371-1372, 1992. Available here.
14. Kannel WB, Gordon T, ed. Section 24: The Framingham Diet Study: Diet and the Regulation of Serum Cholesterol. The Framingham Study—An Epidemiological Investigation of Cardiovascular Disease. Washington, DC: United States Department of Health, Education and Welfare, 1970.
15. Findings of Framingham Diet Study Clarified. The Framingham News, 1970.

All links accessed Jan. 28, 2019.