CrossFit | Morreim Report

Expert Report by E. H. Morreim, JD, Ph.D: CrossFit, Inc., V. National Strength and Conditioning Association

12
ByCrossFitJanuary 27, 2019

Throughout CrossFit’s litigation with the NSCA, the NSCA has consistently defended its decision to coerce and publish fake injury data about CrossFit training by deferring to its purported “rigorous” and “double blind” peer-review process led by William Kraemer. Specifically, the NSCA has claimed—both in public statements and in court filings—that its peer-review process is beyond reproach and facilitates scientific integrity by having objective industry experts review manuscripts and provide comments to the authors (“double blind” is supposed to mean that identities of authors and peer reviewers are hidden from each group).

Before the May 26, 2017, sanctions order finding that the NSCA withheld numerous documents from CrossFit and lied under oath—and also ordering a forensic evaluation to provide CrossFit with a full document production—CrossFit retained E. Haavi Morreim as an expert witness to review the publishing ethics of the NSCA’s behavior during the peer-review process for the Devor Article. Morreim is a professor in the College of Medicine, University of Tennessee. For 20 years, her research and writing have explored medicine’s changing economics, with numerous publications in journals of law, medicine, and ethics.

Morreim issued two reports in the federal action against the NSCA, both of which were drafted before the sanctions order. Her initial report, filed May 8, 2017, and the supplement report, filed Feb. 22, 2018, are provided below.

Morreim will be providing another report once CrossFit receives a complete document production for the neutral forensic evaluator. Morreim has not yet had opportunity to consider all the evidence that led the discovery referee in the NSCA’s recently dismissed state action against CrossFit, Founder Greg Glassman and two CrossFit employees to conclude on Dec. 5, 2017, that:

  • “Dr. Kraemer had been participating in an anti-CrossFit crusade, viewing CrossFit as one of its primary competitors in the fitness industry.”
  • “Dr. Kraemer, as the (NSCA’s) Editor-in-Chief, purposely manipulated events to produce a false and scientifically invalid report injurious to CrossFit and helpful to NSCA.”
  • “CrossFit produced credible evidence that Dr. Kraemer steered the authors of the Devor study to discuss injuries among CrossFit members. Prof. Devor’s first article had no injury data at all. The insertion of injury data came after Dr. Kraemer indicated that such inclusion would make the article publishable.”
  • “When made aware of the clear falsity of the Devor study, Dr. Kraemer failed to take reasonable steps to correct the error and to disseminate the correction. Instead he hid behind the peer review process.”

CrossFit looks forward to Morreim’s forthcoming second supplemental expert report that will address the full scope of suspected NSCA misconduct during the peer-review process for the Devor Article.

Despite lacking more recently discovered evidence when she issued the two reports above, Morreim skillfully delineated how the NSCA’s authors and editors failed to follow well-established ethical standards. First, it is not acceptable for scientists to fabricate data, nor to falsify institutional review board approval, informed consent, and subject supervision. Second, journals are supposed to pursue ethical complaints such as CrossFit’s and to refer credible complaints to the relevant university for investigation. The NSCA failed to appropriately respond to CrossFit’s evidence even after developments demonstrated that its concerns about data fraud were “undeniable.” And not only did the NSCA fail to pursue credible, substantiated evidence that its paper fabricated data, but the NSCA’s late attempt to issue a correction (“erratum”) also exacerbated the damage to CrossFit affiliates’ reputations. As Morreim discussed, NSCA staff were well aware that their correction misled readers by falsely suggesting that training in a CrossFit class had injured two of the subjects. Morreim concluded that the NSCA’s repeated scientific misconduct has been “akin to the academic equivalent of Fake News.”

Comments on Morreim Report

12 Comments

Comment thread URL copied!
Back to 190128
Timothy Noakes
February 6th, 2019 at 4:10 pm
Commented on: Morreim Report

“Bad men need nothing more to compass their ends, than that good men should look on and do nothing.”

John Stuart Mill. Inaugural address at the University of St. Andrews, Scotland. 1867

I chose science as my profession, in part because I believed it to be honourable; a trade in which there is only one goal — the diligent search for a concealed “truth”. And the final revelation of the grand truths often brings great benefit to mankind.

The problem I have learned is that not everyone in science sees it quite that way. And the profession is often reluctant fully to expose those who transgress. So the integrity of science and our profession is further damaged.

The case of CrossFit, Inc.,, v. National Strength and Conditioning Association (NSCA) is a classic example of why some scientific disciplines are in such disarray.

In short the story is the following: A group of scientists from Ohio State University submit the results of their scientific research to the journal whose readership, they believe, will likely be interested in their findings. Their study simply measures physiological training adaptations in CrossFit participants over a 10-week period. The findings are straight forward: Subjects improved key physiologic measures of fitness. The study is not controlled as it does not compare CrossFit exercise to other interventions or to no intervention. It is an uncontrolled observational study in subjects who also ate a low carbohydrate, “paleolithic” diet.

As is the accepted scientific process, under the direction of the Editor-in-Chief of the Journal, Dr William Kraemer, the article is sent out for peer review. The comments of the reviewers are favourable so they can reasonably expect that their work will be published.


However during the review process, as the Editor-in-Chief of the Journal, Dr Kraemer implies that the article needs to include data on injuries for it to be publishable. This is an unusual but, on the surface, not an unreasonable request. It is unusual because the study dealt with physiological changes and the authors may not have sufficient expertise in injury diagnosis and management to provide such information using methods that are reliable and trustable. Not everyone can be an expert in all different areas of sports research; physiology and injuries are usually two separate topics, researched by different groups of scientists.

Nevertheless a subsequent revision of the manuscript is submitted which does indeed include data on injuries that were allegedly sustained by the CrossFit participants during the 10-week training program.

The paper passes additional peer review and is subsequently published (1).

Not long after it is published, it comes to the attention of CrossFit CEO Greg Glassman. He reportedly finds the injury data troubling and institutes an investigation. As a result of his investigation, he chooses to institute legal action against the NSCA.

The expert report by Dr E.H. Morreim explains what has been uncovered so far in the legal actions that have yet to be completed to CrossFit’s satisfaction.

Dr Morreim’s report concludes that CrossFIt have indeed established that the injury data in the paper were fabricated and were added to the manuscript during the period in which the paper was undergoing peer review. Dr Morreim further shows that: “The insertion of the injury data came after Dr Kraemer indicated that such inclusion would make the article publishable”.

Dr Morreim’s report indicates that CrossFit contends that the NSCA has not done enough to mitigate the damage to CrossFit affiliates caused by the publication of this paper. In particular no proper attempt has been made to show that the injury data were fabricated. This requires a specific statement to that effect. But that statement has not been forthcoming. Thus anyone reading the scientific paper will still conclude that CrossFit participants are at high risk of injury and that this scientific paper is the basis for that belief.

Elsewhere I have written about how such a belief can soon become a “Foundation Myth”. Indeed it seems to me that it is not uncommon for persons with no experience of CrossFit to tell me that this is indeed so — that CrossFit carries a high risk of injury.

To make up my own mind on this matter, I read the Devor article (1) to see what I could uncover. I began by reading the authors’ hypothesis which is: “We tested the hypothesis that a 10-week High Intensity Power Training (HIPT) regimen would improve VO2max and body composition in healthy adult volunteers. Furthermore, we hypothesized that improvements of VO2max and body composition would be found across all levels of initial aerobic fitness and body composition, not only in the cohorts of the lowest initial values of these markers”.

So, importantly, the hypothesis contains no evidence that the study set out to establish injury rates in CrossFit participants. If it had, it would have to be in the hypothesis.

But are other pieces of incriminating evidence.


First the abstract contains no mention of injury rates in the participants. This could support a claim that the injury data were added as an after-thought and the authors “forgot” to include that “added” information in the abstract as they re-wrote the article in line with the Kraemer design.


Second the only reference to injury data in the paper is the following sentence: “ Of the 11 subjects who dropped out of the training program, 2 cited time concerns with the remaining 9 subjects (16% of total recruited subjects) citing overuse or injury for failing to complete the program and finish follow-up testing”.


The problem with this information is that it is not included in the Results section of the paper. Instead it is slipped into the Methods section under the topic of Subjects. This is unacceptable and should have been detected in peer review. Results in any study must be included in the Results section, the place where they belong.


However if these results had been fabricated, then this would be an easy way to hide them (as most readers of scientific papers do not read the Methods Section unless they find something unusual that they need to clarify).


But the key point is that from the authors’ position, if the data were indeed fabricated to satisfy their desire to have the paper published, then they had effectively buried the findings out of sight of most readers. As there is no mention in the Abstract and no mention in the Results.

But this argument falls down because the authors clearly did not want to hide the consequences of this “finding”.


For the final two pages of the paper include the following: “Despite a deliberate periodization and supervision of our Crossfit-based training program by certified fitness professionals, a notable percentage of our subjects (16%) did not complete the training program and return for follow-up testing. Although peer-reviewed evidence of injury rates pertaining to high-intensity training programs is sparse, there are emerging reports of increased rates of musculoskeletal and metabolic injury in these programs. This may call into question the risk-benefit ratio for such extreme training programs, as the relatively small aerobic fitness and body composition improvements observed among individuals who are already considered to be ‘above average’ and ‘well-above average’ may not be worth the risk of injury and lost training time”.

If ever there was a scientific non-endorsement of a fitness program, this was it. A classic example of using speculation to undermine what would otherwise be considered a training program that produces highly desirable outcomes. Indeed, what did the authors state in the Abstract: “Our data show that HIPT significantly improves VO2max and body composition in subjects of both genders across all levels of fitness”.

Recall that the Abstract contains no reference to injury or any speculation about how high injury rates would negate these benefits of the CrossFit training program.


The problem with this speculation is that it was never part of the original hypothesis under study. If the goal of the study had been to discover whether the risk of developing an injury outweighs the physiological benefits of CrossFit training, then that should have been the stated hypothesis. But it was not.


Instead what we have here is a classic distortion of the scientific process with ad hoc (after the event) speculations being added that have nothing to do with why the study was originally designed. And which speculations, incidentally, the study cannot answer since it was not designed to research that question.


Thus the sole conclusion one can draw is that pressure was likely brought on the authors to modify their original manuscript in a way that would make CrossFIt training appear less desirable than would have been the case if only the highly beneficial physiological changes in response to training were reported in this article.


The reality is that the inclusion of the fabricated injury data and the resulting speculation in the Discussion section (that the risk of injury in CrossFit participants is high) cannot be allowed to stand. If what we are dealing with is the desire to discover a truth that will be of benefit to many, then the paper must be fully retracted. Or the false sections identified above must be removed.


Until that happens the paper is, as Dr Morreim states, no better than “fake news” that clearly promotes an anti-CrossFit agenda apparently directed, in this instance, by the Editor-in-Chief of the Journal of Strength and Conditioning Research, Professor William Kraemer.


Reference:

1. Smith MM, Sommer AJ, Starkoff BE, et al. Crossfit-based high-intensity power training improves maximal aerobic fitness and body composition. J. Strength Cond. Res. 2013; 27, 3159—3172.

Comment URL copied!
Ginger Sladky
January 31st, 2019 at 8:24 pm
Commented on: Morreim Report

I’m so fucking proud to be CrossFit.

Comment URL copied!
Greg Glassman
January 29th, 2019 at 11:51 am
Commented on: Morreim Report

Where none existed we created a definition of fitness amenable to accurate and precise estimation. This is a prerequisite to measurement, and measurement is necessary to systematically, scientifically, improving fitness. This doesn’t make our definition right or wrong - definitions don’t come in flavors “right” or “wrong” - they’re either consistent or not, useful or not. Defining fitness as work capacity measured across broad time and modal domains is a moment in exercise science.


It also grounds the measurement of fitness to movement and not any one of countless surrogates. We didn’t need university approval for this; we didn’t need any of the academic and “scientific” organizations seemingly chartered to advance fitness (NSCA, ACSM, et al) to approve of our approach. We got the permissions we needed from logic, math, and the scientific method. It turns out that applying Newtonian mechanics to human performance bears fruit as it has in the study of all movement. Who’s surprised? I’m not.


As the principal architect of this approach, I will tell you that my motivation came in great part from the odd and long-nagging realization that exercise science was fruitless. I’ve publicly asked for decades now for someone to name a single contribution to any sports performance or training methodology arising from an advance in exercise science. (Crickets.)


This is the context with which I appreciate, understand, and explain CrossFit - even at Harvard Business School where the subject in my annual visit is our unparalleled business success. I stand in front of business students and explain that we took a stab at the first scientific definition of fitness relying heavily on Sir Isaac Newton and it worked!!


Even our low carb approach to eating had its origins in its dramatic impact measured in force, distance, and time. That is NOT insignificant, even if an aside.


None of this found an audience in academia - a few notable academics excepted. What did get the attention of the academy was our eventual dominance in the trainer certification space and our incestuous (frankly) relationship with first responders and men and women in uniform around the world.


The first significant response from academia was to fake a scientific study. The biggest name in exercise science suborned (were scientific misconduct perjury “suborn” would be the perfect choice of words) scientific misconduct as editor-in-chief of the largest peer-reviewed journal of exercise science and when exposed engaged in perjury and later spoliation of evidence to hide all of the above. “Caught red-handed” would be an understatement. The entire scientific edifice of the NSCA is an out and out fraud.


The corruption came from the top and appears throughout the NSCA. I could let the legal filings, judges’ and referees’ rulings speak for all of this, but I won’t.


We have emails detailing an unholy alliance of the NSCA, Pepsi, Ohio State University, ESPN, and a host of other nationally and internationally renowned companies and organizations coming together to stop what’s going on here on this website and in 15,000 gyms around the world flying the CF flag. What would that effort look like? Here’s just one effort costing millions: a score of bills drafted and floored in a dozen states over a dozen years making CF illegal. I called the NSCA and the ACSM “soda whores”. The NSCA sued me for it. They also dropped the suit after getting caught hiding emails that proved they were, well, soda whores. (On the advice of counsel I won’t mention that Lanny Davis has now come around trying to make this all go away.)


Dr. Morriem’s report was my introduction to scientific misconduct as a field of study. It also introduced me to the “replication crisis” and became the start of a deep and personal academic obsession with coming to understand what is wrong in science and medicine. Our exposure to the scientific fraud at the NSCA has been an education, better yet, a template, for the failures of organized fitness and medicine globally and the lack of meaningful response to the tsunami of chronic disease visible to all of us.


We’ve been to hell and back and didn’t come back empty handed. We owe it to our affiliates, CrossFitters, and the world to share what we've learned and been through.

Comment URL copied!
David Mitchell
January 29th, 2019 at 9:41 pm

Well said Coach. I for one have benefited and am grateful for the work you guys have done in exposing and fighting these absolutely corrupt groups!

Comment URL copied!
Chris Read
January 30th, 2019 at 1:31 am

Thanks for fighting this fight Coach, I am one those who will never participate as an athlete at the CrossFit Games but after 2 years I have lost nearly 70 lbs because of CrossFit and the community at our affiliate!

Comment URL copied!
Jonathan Hurly
January 30th, 2019 at 1:38 am

Thank you. I cannot express this enough... for everything you have done and continue to do, for your eloquence, resilience, and tenacity... thank you.

Comment URL copied!
Christopher Deaton
January 30th, 2019 at 4:41 am

Well said and thank you do not begin to scratch the surface nor will they ever be enough to convey my level appreciation.

Comment URL copied!
Nathan Jenkins
January 31st, 2019 at 10:33 pm

Coach - as I read the report, and your commentary, and then went down a rabbit hole and read various historical documents related to this case (including, for example, the transcript of the phone conversation between Russ Berger and Steven Devor in the CFJ from 2013), I was struck with one thought:


No wonder Greg Glassman is so skeptical of the academy. Anyone in his position would be. You didn't even need it in the first place (defined fitness just fine without our help). When university researchers decided to study CrossFit, this colossal mess was the result. It would make total sense if you never wanted anything to do with academia.


But fast forward ~6 years from the start of the Devor paper debacle and the world is starting to look like a bit of a different place. CrossFit is ubiquitous. Tired questions about injury rates remain, but no one, not even the academics, are arguing over the effectiveness of the methodology for improving fitness and health.


A small but growing number of us in the academy get it. We understand the concept of the fitness-wellness-sickness continuum and that constantly varied high intensity functional movement is probably the optimal prescription for improving health.


We also understand that our scientific literature and our textbooks send a different and wrong message about exercise and health.


My question for you is: what role, if any, do you see for academia in CrossFit's broader mission over the next 5, 10, 15+ years? Should we be doing studies on physiological responses to CrossFit training? Would those add value to the CrossFit Health movement (e.g., to shore up the CrossFit MD's evidence base in the literature, which seems needed to impact standards of care)? What should be our top priorities, from your unique perspective? Would love your thoughts on this.


Alternatively, should we stay out of the CrossFit space altogether, and do our research on something else. Again, I would completely understand that position, given the history.

Comment URL copied!
Allison Autrey
February 1st, 2019 at 6:43 am

Thanks for the persistent fight and exposure Greg! Couldn't be more proud to not only work for CrossFit but be a part of the movement. #truth

Comment URL copied!
Holden MacRae
February 2nd, 2019 at 12:40 am

In addition to the conveyance of misleading, and frankly for the most part trivial research conducted at institutions of higher learning, it is also troubling that medical schools are not held accountable for misguided teaching. Who teaches medical students other than clinical faculty? Most of the teaching at medical schools is conducted by PhD's. Where and by whom are they trained, and, how is the information they present to medical students impacting the quality of health in the US population? Keep in mind that a large subset of medical students were history, philosophy, psychology, fine arts, sociology etc. majors as undergraduates. Yes, they took basic science courses for MCAT preparation, but they were not science majors. If you want to change health outcomes in the US you need to correct the teaching in medical schools by PhD's.

Comment URL copied!
Matthieu Dubreucq
November 11th, 2019 at 8:24 pm

Thanks Coach for taking the time to explain this. It is so important that the public knows what CrossFit stands for and that it isn’t just some pullups and thrusters. You and CrossFit Inc fights to bring the truth and the information and tools needed to take action on what you call the world most vexing problem! Thanks for making us part of the solution.

Comment URL copied!
Tyler Hass
January 29th, 2019 at 4:59 am
Commented on: Morreim Report

This is a very strange case and after reading the reports, I can't help but wonder if the study was just a trojan horse to sneak in a message that CrossFit is dangerous. It can't be cheap to fund a 10-week study with 54 participants. The only questions being studied were the effect of a 10-week CrossFit program on VO2 max and body composition.

As a person interested in both exercise science and CrossFit, this question bores me. There was not even a comparison group. How about the effect of CrossFit training vs a program out of the NSCA manual? And instead of VO2 max, how about power output over a battery of different tests? That would be more interesting to me at least. The Canadian military actually did this type of study and the results were interesting.

The injury data is a mystery. It's odd that it showed up after two rounds of the peer review process but was never mentioned in earlier drafts. William Kraemer seemed to be putting his hand on the scale here and he has enough clout in the field to do exactly that.

Every year, the NSCA honors its top sports scientist with the "William J. Kraemer Outstanding Sports Scientist" award. He's by far the most cited name in the exercise science books I've read. If anyone could exert influence on the direction of a study, it would be him. He was also a co-author in the CHAMP study, so it seems he has an axe to grind against CrossFit.

If this is how he has operated over the course of this career, then it has pretty sad implications for the field of exercise science. He has over 450 published studies under his name and those have been cited 42,000 times (per ResearchGate). Among them, I found this gem: https://renew.advocare.com/health/the-effects-of-hydration-on-performance-importance-of-water-and-electrolytes/

He's got electrolytes!!

Comment URL copied!