Evidence-Based Medicine, Part 3: Can It Be Salvaged?

Bernard Shaw, in his book The Doctor’s Dilemma, written in 1906, outlines a key driver behind much medical activity: “When your child is ill or your wife dying,” when you are confronted by “the spectacle of a fellow creature in pain or peril, what you want is comfort, reassurance, something to clutch at, were it but a straw. This the doctor brings you. You have a wildly urgent feeling that something must be done; and the doctor does something. Sometimes what he does kills the patient.”

Medical history is full of “somethings” that have killed the patient or caused great pain and suffering without providing any benefit at all. Evidence-based medicine grew from the recognition that medicine must become more scientific. The argument was simple: If we are going to do something to a patient, we need evidence that it works, or, at least, that it does more good than harm.

This is a very simple concept and in principle appears impossible to argue against. However, in practice, medicine differs greatly from many other branches of science, not least because medical ethics ensure humans can never be tested to destruction or serious harm; nor can human activities and lifestyle be fully controlled in any clinical trial.

Equally, genetic differences can make a treatment effective for one person and damaging for another. Penicillins are highly effective medications for most people, but in a significant percentage, they can cause anaphylaxis and death.

Young people can react differently to drugs than older people, men differ from women, some individuals have many different underlying medical conditions that can complicate medical treatments (Type 2 diabetes, CVD, COPD, and suchlike), and others do not. Equally, there is a very strong placebo effect that is difficult to control for, and the impact and influence of individual clinicians has a strong effect on patients.

People are not interchangeable units, and how they react can differ greatly depending on an extremely large number of variables. Therefore, evidence gathered from clinical trials on highly selected populations is always going to be, at best, imperfect and cannot possibly be the only factor used to guide treatment decisions.

Unfortunately, in an attempt to create a system of scientific “purity,” the randomized controlled clinical trial (RCT) has become the dominant form of evidence, only surpassed by a meta-analysis of several randomized clinical trials that are all aggregated in an attempt to synthesize (what is perceived to be) the highest-quality evidence.

Gradually, the systematic meta-analysis has taken over as the form of “evidence” that is seen as having greater value than any other, and by a considerable margin. A point argued very forcefully in the Lancet paper “Interpretation of the evidence for the efficacy and safety of statin therapy”: “When large-scale evidence from randomised controlled trials does exist (as it does for statin therapy), the additional value of information from non-randomised observational studies about treatment effects is very limited” (1, my emphasis).

In essence, this article is a long and complex argument, “confirming” that only the data from randomized clinical trials have any real value. Individual patient experiences and observations cannot add any information that would alter clinical judgment.

The dominance of the randomized clinical trial (RCT) has also underpinned the creation of a wide range of guidelines. In one case, this has resulted in the current situation whereby 15 million people are taking statins daily in the U.S. Indeed, were the CVD guidelines to be fully implemented, this figure would be closer to 100 million, including every single man over the age of 60.

This reliance on RCTs, primarily funded by the pharmaceutical industry, has driven a great deal of the research in predetermined directions, primarily in the treatment of common lifelong “conditions” that are considered highly profitable. This, in turn, has resulted in much of the evidence becoming biased, as commercial imperatives distort research findings.

Significant concerns with research have been highlighted repeatedly by the editors of the world’s most influential medical journals. For example, Richard Horton, editor of the Lancet claims:

The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. (2, my emphasis)

His views are supported by Richard Smith, editor of the BMJ for many years, quoting Doug Altman:

Twenty years ago this week the statistician Doug Altman published an editorial in the BMJ arguing that much medical research was of poor quality and misleading … Why, asked Altman, is so much research poor? Because “researchers feel compelled for career reasons to carry out research that they are ill equipped to perform, and nobody stops them.”

“The poor quality of much medical research is widely acknowledged,” wrote Altman, “yet disturbingly the leaders of the medical profession seem only minimally concerned about the problem and make no apparent efforts to find a solution.”

Altman’s conclusion was: “We need less research, better research, and research done for the right reasons. Abandoning using the number of publications as a measure of ability would be a start.”

Sadly, the BMJ could publish this editorial almost unchanged again this week. Small changes might be that ethics committees are now better equipped to detect scientific weakness and more journals employ statisticians. These quality assurance methods don’t, however, seem to be working as much of what is published continues to be misleading and of low quality. Indeed, we now understand that the problem doesn’t arise from amateurs dabbling in research but rather from career researchers.

Smith went on to say:

John Ioannidis showed that almost none of thousands of research reports linking foods to conditions are correct and how around only 1% of thousands of studies linking genes with diseases are reporting linkages that are real. His famous paper “Why most published research findings are false” continues to be the most cited paper of PLoS Medicine.

Ioannidis’s conclusion as to why so much research is poor is similar to that of Altman’s: “Most scientific studies are wrong, and they are wrong because scientists are interested in funding and careers rather than truth.” (3)

This view is reinforced by Marcia Angell, long-time editor of the New England Journal of Medicine, which is considered by many to be the most important medical journal in the world:

It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine. (4)

Unfortunately, it is impossible to disagree with Horton’s statement that “something has gone fundamentally wrong with one of our greatest human creations.” Evidence-based medicine can only possibly be of value if the evidence it rests upon can be trusted. That trust is clearly no longer possible, and we are beginning to see the results.

For many years, life expectancy was rising in all Western countries, but recently, these improvements have stalled and even reversed. As a recent headline in the U.K. highlighted, “Heart disease deaths in under 75’s [are] going up for the first time in 50 years” (5). It seems all the mass medications, guidelines, testing, screening, and scanning is not working anymore.

The same fall in life expectancy can be seen in the U.S. The Atlantic reports:

Health experts typically expect longevity to increase as the economy grows and more health advancements are made, so the fact that life expectancy has been flat or trending downward for years now is concerning.

This data point, says Christopher Murray, the director of the Institute for Health Metrics and Evaluation at the University of Washington, “confirms that there’s a profound change in the trajectory of mortality. This should really be getting everyone’s attention in a major way.” (6)

Of course, there will be reasons other than the failure of evidence-based medicine and the corruption of the research database for this decrease in life expectancy. However, there is clear evidence that the mass medication (polypharmacy) that has followed on from EBM-based guidelines has become damaging rather than beneficial.

An Israeli study attempted to de-prescribe as many medications as possible from an elderly population who were all taking multiple medications. The study was called “Poly de-prescribing to treat polypharmacy: Efficacy and safety.” The study found poly de-prescribing resulted in consistent improvement in quality of life (QoL):

This self-selected sample longitudinal research strongly suggests that the negative, usually invisible effects of polypharmacy are reversible. PDP (poly de-prescribing) is well tolerated and associated with improved clinical outcomes, in comparison with outcomes of older people who adhere to all clinical guidelines and take all medications conventionally. (7)

This builds on other research that demonstrated “PDP was associated with a significant reduction in both mortality and referrals to hospitals, in nursing home patients and with improved clinical outcomes and QoL in community-dwelling older people” (7, my emphasis).

In summary, taking people off several of the medications recommended by guidelines resulted in a significant improvement in quality of life, mortality, and hospital referrals. Turning this evidence around, this would strongly suggest that polypharmacy is a cause of increased mortality and may be a significant factor in the reduction in life expectancy seen in both the U.S. and U.K.

Recall Shaw: “When your child is ill or your wife dying,” when you are confronted by “the spectacle of a fellow creature in pain or peril, what you want is comfort, reassurance, something to clutch at, were it but a straw. This the doctor brings you. You have a wildly urgent feeling that something must be done; and the doctor does something. Sometimes what he does kills the patient.”

Can EBM be salvaged? Only if the public and politicians, and of course doctors, wake up to the fact that “something has gone fundamentally wrong with one of our greatest human creations.”

Additional Reading

• Evidence-Based Medicine, Part 1: Industry Distorts Clinical Priorities
• Evidence-Based Medicine, Part 2: Unpublished Evidence

Malcolm Kendrick is a family practitioner working near Manchester in England. He has a special interest in cardiovascular disease, what causes it, and what may prevent it. He has written three books: The Great Cholesterol Con, Doctoring Data, and A Statin Nation. He has authored several papers in this area and lectures on the subject around the world. He also has a blog, drmalcolmkendrick.org, which stimulates lively debate on a number of different areas of medicine, mainly heart disease.

He is a member of THINCS (The International Network of Cholesterol Sceptics), which is a network of doctors and scientists who believe that cholesterol is not the main underlying cause of heart disease. He remains a proud Scotsman, whisky drinker, and failed fitness fanatic who loves a good scientific debate — in the bar.

References

1. Collins R, et al. Interpretation of the evidence for the efficacy and safety of statin therapy. Lancet 388.10059(Nov. 19, 2016): 2532–2561.
2. Horton R. Offline: What is medicine’s 5 sigma? Lancet 385.9976(April 11, 2015): 1380.
3. Smith R. Medical research—still a scandal. BMJ Op Jan. 31, 2014.
4. Angell M, New York Review of Books, 19 Jan. 2009.
5. Bodkin H. Heart disease deaths in under-75’s going up for the first time in 50 years. The Telegraph. May 13, 2019.
6. Khazan O. Americans are dying even younger. The Atlantic. Nov. 29, 2018.
7. Garfinkel D. Poly-de-prescribing to treat polypharmacy: efficacy and safety. Ther Adv Drug Saf. 9.1(Jan. 2018): 25–43.