Most of what we understand about the safety and harms of different medicines emerges from published drug trials in the peer-reviewed literature, which is often unduly influenced by industry funding, selective reporting, and a lack of data transparency. Maryanne Demasi, Ph.D., discusses how the Statin Wars demonstrate the scope and stakes of this problem.Read MoreWhere Is the Scientific Data Hiding?
HEALTH: SCIENTIFIC MISCONDUCT
CrossFit Health is an investigation into the ills of modern medicine and the wilful abuse of the public’s trust in science. The lessons learned from the legal dismantling of fake science, a crooked journal, and perjuring scientists have given us a forensic view as to how everything might have gone so wrong. We’re calling the combination of runaway medical costs and disease rates — which many profit from but none combat effectively — “The Mess.”
In part two of his series on pathological science, Gary Taubes estimates the extent to which self-deceit has infiltrated scientific research and further investigates whether entire disciplines have become pathological. His inquiries lead to meditations on the necessary characteristics of rigorous scientific study. One indication that health-related research has become pathological, he suggests, is its experts’ tendency to view “placebo-controlled, double-blind, randomized-controlled trials as the ‘gold standard’ of scientific evidence.” This view, he argues, represents “a lack of understanding of the scientific endeavor,” because “these trials are simply what’s necessary to establish reliable knowledge.”Read MorePathological Science, Part 2
In this three-part series, Gary Taubes investigates what Irving Langmuir terms “pathological science,” or the “science of things that aren’t so.” Taubes distinguishes pathological science from fraud, which is characterized by a person’s attempt to deceive others, arguing rather that participants in pathological science delude themselves. He compares this self-deceit to what Richard Feynman famously called "Cargo Cult Science." And while researchers and philosophers of science have long expressed concern over the possibility of self-deceit distorting scientific truth, Taubes questions whether the problem has grown rampant today. “The possibility exists that entire disciplines may be essentially pathological,” he writes.Read MorePathological Science, Part 1
Science is immersed in a crisis of reproducibility, but how can science go so badly wrong? Here, an anonymous author with intimate knowledge of the academic world and its research and publishing practices addresses this question, drawing examples from his formative years in the university and ensuing professional life. Citing firsthand experiences wherein he witnessed the impact of questionable research practices, conflicts of interest, and perverse career incentives, he explains how “the very enterprise of research can deviate systematically from the paths of truth.”Read MoreWhy Do Scientists Cheat?
Gary Taubes is an award-winning investigative science and health journalist, and author of numerous books related to nutrition and the obesity epidemic. In this talk, delivered at the annual CrossFit Health Conference in Madison, Wisconsin, on July 31, 2018, Taubes turns a critical eye toward a more expansive subject, historicizing the corruption of postmodern science and examining the distinctions between good and bad scientific research.Watch Gary Taubes: Postmodern Infection of Science and the Replication Crisis
This 2019 piece documents the work of English anesthetist John Carlisle, who has developed and used statistical methods to identify published research papers with questionable results. Carlisle’s work has found data issues within and outside the anesthesiological research space and has led to high-profile retractions, such as that of the PREDIMED, a study that drove increased interest in the Mediterranean diet in 2013.Read MoreHow a Data Detective Exposed Suspicious Medical Trials
Dr. Maryanne Demasi earned a Ph.D. in rheumatology from the University of Adelaide, but perhaps the most formative experience she had with the medical sciences occurred while she was an investigative journalist with the Australian Broadcasting Corporation (ABC). During her tenure with the ABC, she produced a two-part series called “Heart of the Matter,” which challenged the role of cholesterol in heart disease and addressed the overprescription of statin drugs. The fallout from the series was not swift, but it was decisive. In this presentation, delivered on June 8, 2019, at a CrossFit Health event at CrossFit Headquarters, Demasi shares her personal experiences and the challenges she faced while trying to relay the limitations of statin data to the public.Watch Dr. Maryanne Demasi: My Experience of Exposing the Statin Controversy
Commercial interests have discovered numerous ways to co-opt the loyalty of health professionals to promote company-sponsored studies and marketing messages. Here, Dr. Maryanne Demasi describes just a few of the indications that Australia’s National Heart Foundation (NHF) has sold its soul to industry. She focuses specifically on the NHF’s ties to junk food and pharmaceutical companies, citing responses from various scholars who have expressed deep concern over how the NHF’s receipt of industry funding might threaten public health.Read MoreHas the Australian NHF Sold Its Soul?
In 1993, Prof. Peter C. Gøtzsche co-founded the famous Cochrane Collaboration, an organization formed to conduct systematic reviews of medical research in the interest of promoting unbiased evidence-based science. During his tenure with Cochrane, Gøtzsche fought to uphold its original values. However, when Gøtzsche attempted to correct the path of consensus science or point to industry-related bias, Cochrane sought to censor him and eventually expelled him in 2018 after what he calls a Kafkaesque “show trial.” Here, Gøtzsche shares the research that led to his fallout with Cochrane as well as his firsthand experiences witnessing the organization’s moral collapse.Watch Peter C. Gøtzsche: Death of a Whistleblower and Cochrane’s Moral Collapse
Dr. Glenn Begley, CEO of BioCurate and a respected hematologist and oncologist, is famous for producing a study in 2012 that demonstrated most preclinical research related to cancer drugs cannot be reproduced. In this lecture, he shares the criteria he used for evaluating the research and the methods he found researchers used to distort their data. He then presents his diagnosis of the problem inherent in the academic research cycle, which he claims fuels the desperation that produces such distortions.Watch Dr. Glenn Begley: Perverse Incentives Promote Scientific Laziness, Exaggeration, and Desperation
In the final installment of their series on the sleights of hand that skew scientific research, Drs. Michael and Mary Dan Eades turn a critical eye toward examples of outright fraud. They place studies that stray from the scientific method, such as those published under the supervision of Dr. Brian Wansink, under the rubric of fraud; Wansink encouraged his students to torture data and retroactively create hypotheses to produce publishable papers. The Drs. Eades also cite examples of fraud from the realm of stem-cell research and ultimately ask whether, when published research findings are tainted, it’s possible for physicians to practice evidence-based medicine.Read MoreThe Cardinal Sins of Skewed Research, Part 5: Burning Britches
Drs. Mary Dan and Michael Eades continue their series on the cardinal sins of scientific research, reviewing the bait-and-switch methods researchers often employ to earn or redirect attention. They discuss how statin manufacturers shifted their desired endpoints, turning their focus to cholesterol reduction (regardless of whether such reduction is shown to improve health) rather than the actual prevention of cardiovascular events in order to justify drug prescriptions. The technique of p-value hacking is also considered.Read MoreThe Cardinal Sins of Skewed Research, Part 4: Bait and Switch
Drs. Michael and Mary Dan Eades describe the third cardinal sin of scientific research: sweeping under the rug any negative or unexpected results discovered in the process of testing a hypothesis. Negative and surprising results, they argue, are just as important to the collective advancement of science as positive outcomes. Nevertheless, published research long has reflected a positive outcome bias, and worse, non-reporting often betrays a conflict of interest or self-serving maneuver. They call for “a liberal dose of disinfecting sunlight” on the methods for research reporting and a new broom "that sweeps research back out from under the rug."Read MoreThe Cardinal Sins of Skewed Research, Part 3: Sweeping
Drs. Mary Dan and Michael Eades review the practice of “racking,” one of the cardinal sins of scientific research. In this sin, so termed for its similarity to the practices of medieval inquisitors, scientific data is “tortured until it confesses” a result desired by the researcher (or the researchers’ funder). Among the data manipulations discussed are the selective practices of axis stretching — spacing intervals on a graph to result in a more pronounced visual effect — and the use of relative risk reduction to obscure actual absolute results.Read MoreThe Cardinal Sins of Skewed Research, Part 2: Racking
Scientific inquiry can be dishonestly confounded in a number of ways, and pollution or bias, whether resulting from industry funding or deeply held ideology, is often at the heart of the problem. In this CrossFit Health series, Drs. Michael and Mary Dan Eades review the “cardinal sins” of scientific research, starting with “cloaking,” the sleight of hand that makes the real money behind a study disappear to the casual observer.Read MoreThe Cardinal Sins of Skewed Research, Part 1: Cloaking
“Peer review is a flawed process, full of easily identified defects with little evidence that it works. Nevertheless, it is likely to remain central to science and journals because there is no obvious alternative, and scientists and editors have a continuing belief in peer review. How odd that science should be rooted in belief.” —Richard Smith, former BMJ editorRead MorePeer Review: A Flawed Process at the Heart of Science and Journals
In this piece from 2005, Dr. John P.A. Ioannidis explains that the majority of modern “science” is unscientific. When a range of mitigating factors are accounted for—such as bias, loose interpretations of statistical significance, and small sample sizes—the majority of published “statistically significant” findings are likely to be untrue or unverifiable.Read MoreWhy Most Published Research Findings Are False
"The quality of nutrition papers even in the major scientific and medical journals is so variable and the lack of restraint in the popular media is so great that it is hard to see how the general public or even scientists can find out anything at all." —Richard FeinmanRead the articleReading the Scientific Literature: A Guide to Flawed Studies