HEALTH: DATA / STATISTICS
CrossFit Health is an investigation into the ills of modern medicine and the wilful abuse of the public’s trust in science. The lessons learned from the legal dismantling of fake science, a crooked journal, and perjuring scientists have given us a forensic view as to how everything might have gone so wrong. We’re calling the combination of runaway medical costs and disease rates — which many profit from but none combat effectively — “The Mess.”
The Effect of Statins on Average Survival in Randomised Trials, An Analysis of End Point Postponement
The benefit of preventive drugs such as statins is often quantified as the “number needed to treat,” or NNT, which reflects the number of subjects who would need to be given a drug for one negative clinical event to be prevented. For example, if the NNT for a statin preventing cardiovascular death is 40, it suggests that for every 40 patients given a statin, one cardiovascular death will be prevented. This number, however, distorts the likely reality of the benefit distribution by suggesting a single subject receives the entire benefit while other subjects receive no benefit. This 2014 review aimed to quantify the clinical impact of statins differently, assessing the mean extension of life due to statin treatment. The analysis found the median benefit associated with statin use was 3.2 days in primary prevention and 4.1 days in secondary prevention. In other words, statins extended life expectancy by less than a week. Separate research suggests if the benefits of statins were explained this way, the majority of subjects would not choose to take them.Read MoreThe Effect of Statins on Average Survival in Randomised Trials, An Analysis of End Point Postponement
"Last March, Canada's department of health changed the way it handles the huge amount of data that companies submit when seeking approval for a new drug, biological treatment, or medical device — or a new use for an existing one. For the first time, Health Canada is making large chunks of this information publicly available after it approves or rejects applications. … Researchers who independently re-evaluate drugs say the reports are critical because the data they need is not readily available in medical journal articles. One analysis showed that only about half of clinical trials examined were written up in journals in a timely fashion and a third went unpublished. And when articles are published, they contain much less data than the reports. ... In addition, ‘journal articles emphasize benefits and underplay or, in some cases, even ignore harms’ that can be found in the clinical study report data.”Read the articleCanada's Decision to Make Public More Clinical Trial Data Puts Pressure on FDA
A Comprehensive Review of Randomized Clinical Trials in Three Medical Journals Reveals 396 Medical Reversals
This review found 396 cases between 2003 and 2017 in which a randomized controlled trial indicated an intervention already in clinical use (such as a drug, procedure, vitamin or supplement, device, or more systemic change) lacked an evidential basis. The reversals spread across a spectrum of medical fields, including cardiovascular disease, preventive medicine, and obstetrics and gynecology. This review reinforces the importance of basing clinical recommendations on randomized controlled trial data rather than less rigorous standards of evidence. It indicates a significant share of treatments — even if they are in widespread use — will be found ineffective when tested in a rigorous clinical trial.Read MoreA Comprehensive Review of Randomized Clinical Trials in Three Medical Journals Reveals 396 Medical Reversals
This 2019 piece documents the work of English anesthetist John Carlisle, who has developed and used statistical methods to identify published research papers with questionable results. Carlisle’s work has found data issues within and outside the anesthesiological research space and has led to high-profile retractions, such as that of the PREDIMED, a study that drove increased interest in the Mediterranean diet in 2013.Read MoreHow a Data Detective Exposed Suspicious Medical Trials
Dr. Maryanne Demasi earned a Ph.D. in rheumatology from the University of Adelaide, but perhaps the most formative experience she had with the medical sciences occurred while she was an investigative journalist with the Australian Broadcasting Corporation (ABC). During her tenure with the ABC, she produced a two-part series called “Heart of the Matter,” which challenged the role of cholesterol in heart disease and addressed the overprescription of statin drugs. The fallout from the series was not swift, but it was decisive. In this presentation, delivered on June 8, 2019, at a CrossFit Health event at CrossFit Headquarters, Demasi shares her personal experiences and the challenges she faced while trying to relay the limitations of statin data to the public.Watch Dr. Maryanne Demasi: My Experience of Exposing the Statin Con
The Relationship of Sugar to Population-Level Diabetes Prevalence: An Econometric Analysis of Repeated Cross-Sectional Data
This 2013 analysis, led by Robert Lustig and cited in his lecture at CrossFit HQ on March 9, 2019, investigates the association between sugar availability and diabetes through a review of international food supply data from the U.N. Food and Agricultural Organization. While the analysis is limited by its correlative nature and other issues surrounding both international and observational research, the data indicate that changes in sugar consumption have a stronger association with diabetes prevalence than changes in overall caloric consumption and that, at a population level, sugar consumption and GDP predict diabetes prevalence.Read MoreThe Relationship of Sugar to Population-Level Diabetes Prevalence: An Econometric Analysis of Repeated Cross-Sectional Data
Life expectancy at birth in the United States has fallen for the third year in a row, signaling a crisis in national health. In addition to the opioid epidemic, researchers note statistically significant increases in deaths from chronic diseases such as Alzheimer’s, diabetes, and stroke. In response to this situation, however, U.S. health agencies have understated the problem, ignored the causes, and partnered with the opioid manufacturers, soda companies, and private interests directly contributing to the crisis. A tsunami of chronic disease has landed, and CrossFit affiliates are lifeboats offering a chance for individual health where the institutions established to be the guardians of public health have failed.Read More U.S. Life Expectancy Continues to Decline, Exposing Failure of Public Health Institutions
In this 1975 review, Gina Kolata comments on the cost and necessity of two major clinical trial investments by the National Heart and Lung Institute (NHLI, now the NHLBI): the Lipid Research Clinics Primary Prevention Trial (LRC-PPT) and Multiple Risk Factor Intervention Trial (MRFIT). “With the passage of the Heart, Lung, and Blood Act in 1972, several large-scale clinical trials were planned to see whether people can voluntarily decrease their risk of heart disease. Now, 4 years later, screening for participants in the two most extensive and most expensive of these trials is nearly complete, but the trials are turning out to cost far more than anyone anticipated,” Kolata writes. As she notes, “A national obsession with dietary fats and cholesterol seems to have developed despite the fact that there is as yet no conclusive evidence that people can voluntarily decrease their risks of heart attacks by changing their diets.” Nevertheless, the LRC-PPT and MRFIT were seen as significant contributions to the diet-heart hypothesis and reflect the commitment on the part of research funders to find this missing evidence.Read MorePrevention of Heart Disease: Clinical Trials at What Cost?
Drs. Mary Dan and Michael Eades review the practice of “racking,” one of the cardinal sins of scientific research. In this sin, so termed for its similarity to the practices of medieval inquisitors, scientific data is “tortured until it confesses” a result desired by the researcher (or the researchers’ funder). Among the data manipulations discussed are the selective practices of axis stretching—spacing intervals on a graph to result in a more pronounced visual effect—and the use of relative risk reduction to obscure actual absolute results.Read MoreThe Cardinal Sins of Skewed Research, Part 2: Racking
The WHO trial (so named because the international team of principal investigators contained World Health Organization members) tested the potential of clofibrate, a “pre-statin” cholesterol-lowering agent, to reduce heart attack morbidity and mortality. The investigators ultimately concluded that clofibrate "cannot be recommended as a lipid-lowering drug for community-wide primary prevention of ischaemic heart disease.” Nevertheless, clofibrate remained in use until 2002, when it was pulled for increasing cancer rates. In their review of studies such as the WHO trial, Uffe Ravnskov and David Diamond observe, “Despite the largely disappointing findings from 50 years of cholesterol lower[ing] trials, the indictment and conviction of cholesterol as the causal agent in CVD [cardiovascular disease] has stood the test of time. … [Yet] the grand effort to reduce cholesterol as a strategy to improve health has failed.”Read More The WHO Trial
"P values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume."Read the articleScientific Method: Statistical Errors
“If the statement succeeds in its purpose, we will know it because journals will stop using statistical significance to determine whether to accept an article. Instead, journals will be accepting papers based on clear and detailed description of the study design, execution, and analysis, having conclusions that are based on valid statistical interpretations and scientific arguments, and reported transparently and thoroughly enough to be rigorously scrutinized by others.” —Ron Wasserstein, Executive Director, American Statistical AssociationRead MoreWe’re Using a Common Statistical Test All Wrong. Statisticians Want to Fix That.
In this piece from 2005, Dr. John P.A. Ioannidis explains that the majority of modern “science” is unscientific. When a range of mitigating factors are accounted for—such as bias, loose interpretations of statistical significance, and small sample sizes—the majority of published “statistically significant” findings are likely to be untrue or unverifiable.Read MoreWhy Most Published Research Findings Are False
“Framingham investigators seemed to be objective in their early reports but subsequently exhibited stronger and stronger biases. The use of such terms as ‘powerful’ to describe an exceedingly weak relationship has been commonplace. One kind of bias exhibited was the process of reinterpreting data in order to be consistent with the current position of the alliance (the group providing the funding; i.e., the NHLBI).” —Russell SmithRead MoreThe Framingham Heart Study, Part 3: Framingham’s Presentational Flaws—Bias or Fraud?
“Problems associated with the [Framingham Heart] study may be categorized as 1. methodological, 2. analytical, 3. presentational, and 4. interpretation error. The first problem can generate error independent of the investigators, while the latter three problems can be or are definitely generated by the study investigators.” —Russell SmithRead MoreThe Framingham Heart Study, Part 2: The Framingham Observation
"The mission of the Cochrane Collaboration, established in 1993, was to systematically review medical evidence with a view to producing the best quality and trustworthy evidence. Twenty-five years later, it is in a crisis that centres on the dismissal (of) one of its founders and the question of access to clinical trial data. The original mission aimed at improving health. In the face of stalling life expectancies, the stakes in the current crisis could not be higher." —David HealyRead MoreThe Crisis in Cochrane
“The Framingham Heart Study laid the groundwork for the obsession we’ve had with cholesterol and saturated fat and may well be the study that has been most damaging to the health of the U.S. population. This damage comes not necessarily from the study’s data but from the misreporting, deception, dissembling and outright prevarication about the data that have made it the wellspring for both the diabetes and obesity epidemics that afflict us.” —Dr. Michael EadesRead MoreThe Framingham Heart Study, Part 1: Cargo Cult Science