CrossFit | The secret life of data

The Secret Life of Data

ByMaryanne Demasi, Ph.D.July 26, 2019

In 2013, I was investigating the clinical trial data behind statins, the world’s most widely prescribed medications for lowering cholesterol.

I discovered the raw data on the clinical trials of statin drugs had been kept secret, held by a group of researchers called the Cholesterol Treatment Trialists (CTT), based at Oxford University.

The CTT, headed by Professor Colin Baigent and overseen by Professor Sir Rory Collins of the Clinical Trial Service Unit (CTSU), periodically releases meta-analyses of the trials, advocating the wider use of statins in the population, including among people with no symptoms of heart disease.

Independent researchers have never been able to authenticate the CTT’s conclusions because no one else has had access to the data. This has created considerable unrest, since the bedrock of science is contestability and replicability.

Six years ago, I wrote to the CTT to inquire about access to the raw data on statins.  After a lot of toing and froing, Baigent finally conceded that the CTT could not authorize my request.

Baigent wrote, “The CTT secretariat has an agreement with the principal investigators of the trials and … with the companies themselves, that individual trial data will not be released to third parties … . Such an agreement was necessary in order that analyses of the totality of the available trial data could be conducted by the CTT Collaboration: without such an agreement the trial data could not have been brought together for systematic analysis” (my emphasis).

So, there it is — a legally binding instrument that prevents this group of researchers from sharing data that would enable independent analysis.

There have been many initiatives to demystify the secrecy surrounding clinical trial data, but one stands out.  Work led by Danish physician Peter Gøtzsche resulted in a landmark decision by the European Medicines Agency (EMA) to allow public access to “any document originated, received or held by” the drug regulator.

This meant that a wide range of regulatory documents, including clinical study reports (CSRs) and protocols, became available to anyone. CSRs, which are hundreds or thousands of pages in length, contain a hidden trove of unpublished data on medications.

A group of researchers took advantage of this untapped resource and gathered all the regulatory documents relating to antiviral drugs (likeTamiflu). Their discovery shook the scientific world after it was revealed that the totality of trial data (much of which had never been made public) showed that antivirals were far less effective than governments and doctors had previously thought.

Since then, there have been multiple examples of how meta-analyses of CSRs yield substantially different results compared to published trials in peer-reviewed journals.

Another significant moment in scientific transparency occurred recently.  Dr. Peter Doshi, Assistant Professor at the University of Maryland, requested a series of trial documents from the Canadian drug regulator Health Canada so his team could undertake an independent analysis of its data.

Initially, Doshi was denied access to the documents unless he signed a confidentiality agreement that would prevent him from making his findings public. But Doshi filed a lawsuit in a federal court seeking a judicial review of the refusal.

In 2018, a federal court judge ruled in favor of the public’s interest and ordered Health Canada to grant Doshi access to the trial documents, a decision that has paved the way for greater scientific transparency.

My area of interest has been in statin drug trials. The CTT has analyzed data on deaths, vascular events, and cancers but not on low-grade myopathy or muscular harms. This has generated huge debate, because the rate of low-grade myopathy documented in controlled trials is at odds with the high rate seen in observational studies.

Fiona Godlee, the editor-in-chief of the BMJ, has already called for an independent inquiry into the statin data. We have joined this initiative. Now that the EMA and Health Canada have opened access to CSRs and protocols, we have an opportunity to bypass the CTT and obtain the data we require from drug regulatory agencies.

We have already begun this process by systematically documenting which national drug regulators across Europe and North America have licensed statins and which hold the trial evidence upon which these drugs were approved.

We reason that analyzing the regulatory data on statin harms in primary prevention (asymptomatic population) will enable a more considered debate about these important public health drugs. Stay tuned.

Dr. Maryanne Demasi is a well-known investigative journalist whose work on scientific documentaries has been praised by the National Press Club of Australia for exhibiting “excellence in health journalism.”

Demasi earned a Ph.D. in rheumatology from the University of Adelaide in 2004. She currently works as a researcher for the Nordic Cochrane Centre.

Comments on The secret life of data


Comment thread URL copied!
Back to 190727
Richard Sheehan
July 27th, 2019 at 1:01 am
Commented on: The secret life of data

MaryAnne, we met in Santa Cruz a few weeks back at a Crossfit DDC. I understand the importance of proprietary data and use it regularly. And it generally comes with restrictions on its use and sharing. In finance and economics, there are strong requirements for sharing the data for any study and that requirement runs across virtually all journals. In many cases, the data must be placed in a repository of the journal. The only generally accepted exception to the rule is for proprietary data. So I'm not surprised that they refused to provide the data. However, again in finance and economics, even if you cannot share the data you are expected to share any and all programs and code used to analyze the data. That's a far cry from having the data itself, but having the programs, code and raw results from the data could be nearly as informative as the underlying data. At a minimum, you should be able to determine things like whether the sample sizes were appropriate, whether there were outliers removed or other manipulations made to the data and whether those manipulations may have compromised the results, and whether the results were statistically significant and medically or economically significant. It should be possible to review those documents without compromising the proprietary data. If they cannot make that documentation available, then I would argue that there's likely a problem with the underlying analysis or that they simply don't want to expose their study to a rigorous external review. Either way, it would appear to be a serious breach of scientific protocol. Good luck!

Comment URL copied!