“What I’m alleging is that the reviewers, the editors of the so-called top-tier journals, grant review committees, promotion committees, and the scientific community repeatedly tolerate poor-quality science,” said Dr. Glenn Begley during a CrossFit Health event at CrossFit Headquarters on June 8, 2019.
Begley, CEO of BioCurate and a respected hematologist and oncologist, is famous for producing a study in 2012 that demonstrated most preclinical research related to cancer drugs cannot be reproduced. In this lecture, he shares the criteria he used for evaluating the research and the methods he found researchers used to distort their data. He then presents his diagnosis of the problem inherent in the academic research cycle, which he claims fuels the desperation that produces such distortions.
Begley’s six criteria for assessing preclinical research included the following questions:
- Were studies blinded?
- Were all results shown?
- Were experiments repeated?
- Were positive and negative controls shown?
- Were reagents validated?
- Was the analysis appropriate?
Through his analysis of research published in top-tier journals, Begley found that most data appeared to be distorted in order to produce a finding that the researchers had decided upon a priori.
“It’s easier to get the results you want if you know what you’re looking for,” Begley explains.
He highlights several methods the researchers he analyzed used to strategically present their data and exaggerate its significance. These methods included varying degrees of obfuscation, from selectively sharing results to representing cell growth as linear rather than exponential or logarithmic.
Begley refrains from characterizing such practices as outright fraud, instead opting to see them as symptoms of a systemic problem. “We get what we incentivize,” he says. The problem with scientific research, he argues, relates to problems with the academic research cycle and its promotion of perverse incentives that bear no relation to quality.
Despite his ominous portrayal of the state of laboratory science, Begley remains optimistic. “Science is the most effective tool humans have ever developed for driving progress,” he explains. “We’ve made enormous advances, but we could have done so much better,” he adds later.
To read a full transcription of the lecture, click here.
Comments on Dr. Glenn Begley: Perverse Incentives Promote Scientific Laziness, Exaggeration, and Desperation
Finally getting around to watching this. I am always eager to watch presenters go through a Q&A because I think it kind of shows how they are when things aren't going according to the script, and I I especially liked how Dr. Begley came across in the Q&A segment here. Fair, open-minded, and, ultimately, very real. Thanks to everyone who set this event up, and thanks to CrossFit for keeping these lectures available for reference.
Great example and I particularly like the position of Dr. Glenn Begley on fraud v.s. laziness. This is probably the best way to approach the scientific community if we want them to hear the message.
Dr. Begler saying “It’s easier to get the results you want if you know what you’re looking for,” reminds me of the CrossFit Health Post on Coca-Cola funded research, in that Coke has control of the analysis of the research the fund. So we're coming full circle here, the entire community is strapped for cash and pushing out new findings is how you get it. Gone are the days of reproducing others' findings, it doesn't have the appeal and novelty that gets the research dollars. It's critical that findings can be reproducible, otherwise, they are not truly new findings, just one-off results do to randomness at best and bias at worst.
We were lucky enough to have been able to see Dr. Begley's revealing (and sadly troubling) talk. It's a clear, concise, and up close examination of the mess that is medical/pharmaceutical/nutritional research that points the finger where it belongs -- at the publish or perish system that fuel the scientific equivalent of Gresham's Law: bad science drives out good.
Dr. Glenn Begley: Perverse Incentives Promote Scientific Laziness, Exaggeration, and Desperation
4